Maintenance30 points
Whether the code is still moving, and whether more than one person is moving it.
HOW THE POINTS ARE AWARDED
- 1Recency of the last commit, 15 points inside a month, 12 inside three, 8 inside six, 4 inside a year, 0 after that.
- 2Tagged releases in the last twelve months, 10 points at four or more, sliding to 0 at none.
- 3Contributor count, 5 points at twenty or more, sliding to 0 at none.
WHERE IT CANNOT BE MEASURED
A kit with no public repository scores nothing here, because there is no public surface on which its maintenance can be observed. A vendor changelog is a summary the vendor writes about itself, which is a different kind of thing from a commit history.
Completeness25 points
How many of the fourteen things people buy a starter kit for are actually in it.
HOW THE POINTS ARE AWARDED
- 1For an open-source kit, each capability is looked for in the dependency manifest and the file tree, what the kit ships, not what its README says.
- 2For a closed kit, each capability is looked for on the vendor's own feature and documentation pages, and the sentence it was found in is recorded.
- 3The score is the share of the fourteen that were found, out of 25 points.
Transparency20 points
How much a buyer can check before paying.
HOW THE POINTS ARE AWARDED
- 1A public source repository, 6 points.
- 2A stated licence, 4 points.
- 3A price published on a page that can be read without an account, 4 points.
- 4A public release history, 3 points.
- 5Public documentation, 3 points.
Documentation15 points
Whether the documentation exists, and how much of it there is.
HOW THE POINTS ARE AWARDED
- 1A documentation site, 7 points, plus up to 8 more for its depth, counted as the number of distinct pages linked from its index.
- 2A README long enough to be the documentation and no site, 5 points.
- 3Neither, 0.
- 4A documentation site whose navigation only exists after JavaScript runs cannot have its depth counted, so it keeps the 7 and scores no depth. That is a limit of the measurement and is shown as one.
Support10 points
Whether there is a stated way to get help.
HOW THE POINTS ARE AWARDED
- 1A private or priority channel stated publicly, 10 points.
- 2A community channel or issue tracker, 6 points.
- 3Nothing stated, 0.
What cannot be measured16 kits
A closed kit with no public repository has no surface on which its maintenance can be observed. That is a fact about the evidence, not a fact about the kit, and the two are never printed as the same thing here. The component is marked unmeasurable, its points stay in the denominator, and a kit’s score is still out of a full 100. A scorecard shows an unmeasurable component as a dotted span on the rule, never as an empty one.
16 of the 38 kits in the index have at least one component that cannot be measured, and every one of those pages says which component and why.
Run here, or read16 run
Search for a SaaS starter kit and the results are vendors reviewing themselves and affiliates reviewing their own commissions, all written in the voice of somebody who used the thing. Almost none of them ran it. Every row here says plainly which kits were installed and built and which were only read about, and that distinction, not the ordering, is what makes the rest of the numbers worth anything.
16 kits were cloned from a clean checkout and built, with every command, its exit status and its duration published on the kit’s own page. 22 were read from their repository, their dependency manifest and their own documentation. A failing build is not held against the score: most starter kits need a database URL before they will build.