For teams whose commits fail on red tests, Greencently offers an optimization: the pre-commit hook can know when you’ve run all tests green “recently enough”. Team chooses policy.

Benefits (immediate)

• Lower-friction commits
• Time and energy saved

Benefits (speculative)

Probably you’ll lean into smaller, more frequent commits. Maybe you’ll also feel more able to invest in further test speedups.

Usage

Nobody needs to learn anything. The pre-commit hook still has your back when you forget to…

• Run any tests
• Run all tests
• Attend to red tests

With Greencently, the pre-commit hook also has your back when you remember.

All tests green, recently? Commit quickly and stay in flow.

Setup

Available for JUnit 5, so far. More implementations welcomed (see FAQ, below).

1. Append to .gitignore: *when-all-tests-were-green*
2. Fetch latest version number from Maven Central:

   $ curl --silent 'https://search.maven.org/solrsearch/select?q=com.schmonz.junit-greencently&wt=json' \
   | jq -r '.response.docs[0].latestVersion'
   

3. Update build.gradle.kts:

   dependencies {
    testRuntimeOnly("com.schmonz:junit-greencently:VERSION_NUMBER_HERE")
   }
   tasks.withType<Test> {
    jvmArgs("-Djunit.jupiter.extensions.autodetection.enabled=true")
    maxParallelForks = 1  // until we know how to combine results from Gradle's Test Executors
   }
   

4. Run all tests for project
5. If green, observe .when-all-tests-were-green-junit5 at top level of repo (and not in git status)
6. Inspect file modification time in pre-commit hook, perhaps like so:

   #!/bin/sh
   all_tests_were_recently_green() {
    thenstamp=$(date -r .when-all-tests-were-green-junit5 '+%s' 2>/dev/null || echo 0)
    nowstamp=$(date '+%s')
    secondsago=$(expr ${nowstamp} - ${thenstamp})
    [ ${secondsago} -lt 30 ]
   }
   if all_tests_were_recently_green; then
    ./gradlew clean build -x test
   else
    ./gradlew clean build
   fi
   

Reasons not to bother

No appreciable benefit if…

• You’re not using JUnit 5 (for now — other integrations are very desired), and/or
• You don’t have or want a pre-commit hook that runs the tests, and/or
• Your complete test suite runs in microseconds (nice!), and/or
• You commit 2-3 times per day and that’s totally fine

Bad tradeoff if…

• In your context it somehow adds more risk than it mitigates, and/or
• The behavior change you want to see first is something other than “smaller, more frequent commits”, and/or
• You were hoping to introduce something revolutionary, or at least challenging for people to adjust to

Frequently Asked Questions

1. What problem are you trying to solve?

A small one: running the same tests on the same code a second time offers no marginal benefit, costs twice as much, interrupts flow, discourages frequent small well-tested commits, and engenders learned helplessness.

2. Why are you trying to solve it?

Hm, I guess it’s not that small. Plus it’s experienced frequently by many people.

3. What’s this scenario where tests are getting run twice on the same code?

1. A conscientious developer runs them just before committing, to check their work
2. A pre-commit hook runs them just before committing, to check their work

4. How does Greencently help?

It gives your pre-commit hook a way to skip running the tests when you just ran them.

5. Why not just turn off the pre-commit hook?

Because we’re people, every now and then we might forget to run the tests at the right time. And because we’re conscientious people, we change our environment to contain the damage.

6. What’s so bad about running the tests twice?

What’s so good about it? Is there honor in waiting longer for no additional feedback? Or do we only trust our tests after they’ve been green several times in a row?

7. Why aren’t you telling people to make their tests faster?

Are we worried that if tests are slow only once per commit, we won’t care enough to do anything about it?

In my experience as a software development coach, most teams and codebases don’t have very fast tests, for entirely understandable reasons, and can’t easily have them by tomorrow. Improving the situation takes time, attention, and shared expertise. Everyone likes fast tests — once they’ve experienced them.

By contrast, spend a few minutes adding a tiny JUnit extension to your project in the morning, and by the end of the day you’ll probably have saved more minutes than that. Multiply over time and teammates, and maybe now you can afford to invest in the cost-effectiveness of your test designs.

8. Doesn’t this introduce risk?

Yes, but assuming your build pipeline also runs the tests (right?), there’s no new hole here — see the “Risk” aside for details.

There’s no risk-free choice, either:

• If commits are safe because they’re automatically tested, but the tests are (doubly) slow, people will tend to make commits that are bigger and riskier
• If commits are fast because nothing causes tests to get run, people might make mistakes they wish they hadn’t
• If commits are fast and safe, then decisions are currently being made to invest in cost-effective tests, but those decisions rarely persist as long as the tests do

Software developers everywhere need support to break the falsely held dichotomy between safety and speed. Greencently provides immediate relief, requires zero organizational change, and incentivizes behavioral change. Even it’s a small step, it’s probably in the right direction.

9. How else might one implement this idea?

Actual: Gradle build cache

If you run ./gradlew test all green and then run it again without having changed anything, Gradle knows not to run the tests again.

Compared to Greencently:

• yay: no additional pre-commit logic
• yay: no time window for changing what’s being committed out from under the tests
• nay: often much slower to decide than IntelliJ’s internal test runner

Actual: IntelliJ Run Configurations

A clever Run Configuration can do the trick, via @rradczewski:

1. Create a Shell Script configuration “Clever Test Task”
2. Execute script text touch .when-all-tests-were-green-intellij
3. Add a “Before launch” task -> Run Another Configuration -> your existing all-tests task (with “Activate tool window” selected, so you see the output)
4. Whenever you’d run all tests, run Clever Test Task instead

If and only if the “Before launch” task succeeds, the shell script will run.

Compared to Greencently:

• yay: cool hack and effective use of IDE
• nay: tests must run within IDE to yield a commit speedup

Imagined: Digging into IntelliJ test history

IntelliJ has Run -> Test History, but the on-disk representation is not updated anywhere near real time. If it did, the following command would approximately get you to the next problem, which is deducing all-green or not from the results:

$ xq '.project.component[] | select(.["@name"] == "TestHistory") | .["history-entry"][]["@file"]' \
      < workspace/something-or-other.xml \
      | grep ^\"All

Imagined: Digging into JUnit test reports

If JUnit is configured to generate reports, you could try to determine from the content of a report whether that all possible tests ran and all of them were green, and if so, check the file’s timestamp. Likewise with the Surefire reports if running under Maven with Surefire.

Imagined: Arranging particular JUnit test-run ordering and output

Perhaps a pseudo-test could be arranged to run only when the whole suite runs, to always run last, and to introspect the results of the other tests.

10. What are some neighboring ideas?

test && commit || revert

AKA ”TCR”, from Kent Beck, about manually deciding when to run the tests and automatically deciding the rest: auto-commit if green, auto-revert if red.

Compared to TCR, Greencently is less interesting, but also a less radical step from most folks’ existing workflow. TCR might be an experiment that only a small subset of developers are positioned to try; Greencently might be a small optimization that many developers can easily adopt.

Can you add Greencently to your TCR setup? Probably not with a pre-commit hook, but could fit nicely in cahoots with pre-push.

Infinitest

Infinitest is an IDE plugin that observes the code being changed, deduces which tests could be affected, and continuously runs those.

Infinitest provides more timely code-writing feedback than running tests manually later. Greencently doesn’t do anything to shorten that feedback loop, but it does shorten the slightly broader loop that includes commits, and it accomplishes this by avoiding a redundant (i.e., feedback-free) step.

Can you use Greencently and Infinitest together? Definitely. They’re orthogonal: Infinitest will (probably) never run all the tests, but if it does, Greencently will (probably) save you a little time.

make

The job of a build tool like make (or gradle, et al.) is to hold a single complete map of the dependencies between source files and generated files, along with instructions for turning the former into the latter. With a well-written Makefile, you can define a target that runs the tests if and only if any source files have changed.

11. How can I help with Greencently?

Simplify Gradle setup

Figure out how to aggregate results from parallel Test Executors, so we can stop needing to force maxParallel to 1. (Yep, performance will probably improve too.)

Document Maven setup

Greencently ought to work with Maven’s test runner. Try it and let me know how it works (or doesn’t).

Support more test frameworks

For JVM-hosted frameworks such as JUnit 4 and TestNG, let’s extend junit-greencently. For instance, to add JUnit 4 support, we’ll need:

1. JUnit4Planner that discovers and enumerates all tests in a given project the same way JUnit 4 itself does (ideally via a public API call into the framework)
2. JUnit4Summary that reports results of a test run
3. JUnit4Listener that registers callbacks (for each test and for the entire test run), then compares a JUnit4Summary of the tests we planned with a JUnit4Summary of the tests that actually ran

For non-JVM frameworks such as Jest, RSpec, CMake, GoogleTest, Unity, and CppUTest, we may or may not want separate repositories (you can help me decide), but we’ll probably want similar abstractions.

Integrate directly with IDEs

IntelliJ, for instance, has a well optimized test runner. An IntelliJ plugin could probably run very efficiently.

For teams living in IntelliJ, the IDE plugin might be all they need. For teams who also sometimes run tests from the command line, no harm in also configuring Gradle to load the JUnit extension.

Add tests

• Do we observe when some tests (but not all) ran, and correctly ignore?
• Do we observe when any tests were red, and correctly ignore?
• Do we observe when all tests ran and all were green, and correctly act?
• Is JUnit5Planner counting “dynamic tests” the same as JUnit 5?