dart-lang / native / 12066962280

92%

Build Type

push

github

Committed by web-flow

Commit Message

[native_assets_builder] Use file content hashing (#1750)

This PR changes the caching behavior for hooks to be file content hashing instead of last modified timestamps.

Closes: https://github.com/dart-lang/native/issues/1593.

In addition to using file hashes, a timestamp is passed in to detect if files were modified during a build. The moment of hashing contents is after a build is finished, but we should consider files changed after the build started to invalidate the build. If this happens, the build succeeds, but the cache is invalidated and a warning is printed to the logger.

The implementation was modeled after the [`FileStore` in flutter_tools](https://github.com/flutter/flutter/blob/1e824af6b/packages/flutter_tools/lib/src/build_system/file_store.dart). However, it was adapted to support caching directories and to match the code style in this repository.

Directory caching is defined as follows: the hash of the names of the children. This excludes recursive descendants, and excludes the contents of children. For recursive caching (and glob patterns), the populator of the cache should do the glob/recursion to add all directories and files.

### Testing

* The existing caching tests (such as `pkgs/native_assets_builder/test/build_runner/build_runner_caching_test.dart`) cover caching behavior.
* Now that last-modified are no longer used, some sleeps have been removed from tests. 🎉 

### Performance

Adding a stopwatch to pkgs/native_assets_builder/test/build_runner/build_runner_caching_test.dart for the second invocation of the hooks so that it is cached. 

* `lastModified` timestamps: 0.028 seconds (pre this PR)
* `package:crypto` `md5`: 0.047 seconds (current PR)
* `package:xxh3` `xxh3`: 0.042 seconds

The implementation does not use parallel system IO for loading files (no `Future.wait`), but does use async I/O to allow flutter_tools to run other `Target`s in parallel.

The (pre and ... (continued)

Run Details

99 of 103 new or added lines in 2 files covered. (96.12%)

12 existing lines in 2 files now uncovered.

11483 of 12924 relevant lines covered (88.85%)

35.44 hits per line