Repository navigation
Support for Caching on Web using IndexedDB - #504
ben-milanko wants to merge 25 commits into
Conversation
update, test stub
|
Tested it and it works very well on my app. |
…ync-and-deps # Conflicts: # flutter_cache_manager/example/macos/Flutter/GeneratedPluginRegistrant.swift # flutter_cache_manager/example/pubspec.yaml # flutter_cache_manager/lib/src/config/_config_web.dart # flutter_cache_manager/pubspec.yaml
Raise direct dependency floors to the latest published versions (clock 1.1.3, path_provider 2.1.6, sqflite 2.4.3, build_runner 2.16.1, mockito 5.8.1, web 1.1.1) and record the fork's IndexedDB web support alongside the upstream sync in the changelog. Also picks up the analyzer excludes the Flutter 3.47 toolchain writes into analysis_options.yaml on pub get. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCmKSRuHNETmfGu2TffVzo
Upstream CI now runs `dart format --set-exit-if-changed .`, and the fork's IndexedDB repository, connection pool, file system and example were still in the pre-3.7 style. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCmKSRuHNETmfGu2TffVzo
CI pinned Flutter 3.44.4 while the packages now build against 3.47.1. Raising the pin surfaces two use_super_parameters findings in upstream code, so convert both constructors to super parameters and drop the web_helper import that becomes redundant as a result. `flutter analyze` is clean for both packages on 3.47.1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCmKSRuHNETmfGu2TffVzo
- Introduced `CancellationToken` class to manage request cancellations. - Updated `CacheManager`, `BaseCacheManager`, and `FileService` interfaces to accept `CancellationToken`. - Enhanced file fetching methods to throw `CancelledException` when requests are cancelled. - Added tests for cancellation functionality in `HttpFileService` and `CancellationToken` behavior.
- Updated `ImageCacheManager` methods to accept `CancellationToken` for file fetching operations. - Ensured cancellation support is integrated into both direct file retrieval and resized image handling.
- Changed the constructor to ensure a default http.Client is used if none is provided. - Updated request handling to utilize AbortableRequest for better cancellation support. - Modified tests to reflect the new behavior, ensuring that cancellation after response completion does not throw an exception.
- Added checks to prevent operations on already cancelled requests in `CancellationToken`. - Enhanced `WebHelper` to handle cancellation before starting downloads and during queue processing. - Ensured proper cleanup of subjects associated with cancelled requests to prevent memory leaks.
- Added multiple cancellation checks throughout the file update process in `WebHelper`. - Ensured that cancellation is checked before and after asynchronous operations, including downloads and file saves. - Updated `_manageResponse` and `_saveFile` methods to accept `CancellationToken` for improved cancellation support.
Rebuilds test/mock.mocks.dart against mockito 5.8.1 and applies dart format to the cancellation changes so CI's format check passes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCmKSRuHNETmfGu2TffVzo
…am-sync-and-deps Sync with Baseflow upstream and upgrade dependencies
flutter_cache_manager_firebase depends on flutter_cache_manager ^3.4.2 from pub.dev, which has no CancellationToken. FirebaseHttpFileService overrides HttpFileService.get, so once that signature gained a cancellationToken parameter the package stopped analyzing: error - Undefined class 'CancellationToken' error - The named parameter 'cancellationToken' isn't defined Override the dependency with the sibling package in this repository, so the two move together. Overrides are ignored by consumers, so this does not affect anyone depending on the published package. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TCmKSRuHNETmfGu2TffVzo
…-request-cancellation-2 Network request cancellation (rebased onto the upstream sync)
Retain RES dependency layout and check cancellation after awaited persistence. Validated 118 core tests and Firebase test. Co-authored-by: ben-milanko <20643387+ben-milanko@users.noreply.github.com>
rickdijk
left a comment
There was a problem hiding this comment.
Thanks for putting this together and for keeping it in sync with main. People have asked for persistent caching on web many times (#480, #339, #361, #122), and cancellation would be useful too, so we'd like to get both in. We can't merge this PR as it is, though. Format, analyze and tests all pass, but the issues below don't show up in those checks. I've put a suggested way forward at the end.
Breaking changes
This would go out as a minor release (3.5.1), but it breaks existing code.
file_system.dart is no longer exported from package:flutter_cache_manager/flutter_cache_manager.dart, so apps that pass IOFileSystem, MemoryCacheSystem or their own FileSystem to Config stop compiling.
FileService.get has a new parameter. Any existing FileService subclass that overrides get becomes an invalid override, and so does any class implementing BaseCacheManager. Our own published flutter_cache_manager_firebase is one of them: it allows core ^3.4.2, so anyone who upgrades core gets a compile error. The dependency_overrides in the Firebase pubspec hides this inside the repo, but users don't get that override.
The web default also changes from in-memory storage to IndexedDB. Every web app would start using persistent browser storage after a minor update, and the README doesn't mention it.
Changes that shouldn't be in this PR
The branch is the fork's develop, so it carries changes that undo how this repo works now:
- It deletes
AGENTS.mdandCLAUDE.md. - It reverts
CONTRIBUTING.mdand the PR template to the olddevelop/[Unreleased]flow and addsdevelopto the workflow triggers.mainis the only long-lived branch here. - It removes the "Set up Flutter" step from both publish jobs. Both packages depend on the Flutter SDK, so
dart pub publishwould fail and we couldn't release. - It lowers the CI Flutter pin from 3.47.4 to 3.47.1.
- It changes the example image URL and adds analyzer excludes, neither of which relates to the features.
- In
CHANGELOG.md, the version skips 3.5.0, the new entry describes the fork sync rather than the features, and a "Fork (Railway-Engineering-Solutions)" section is added to the 3.4.3 entry, which has already been released.
IndexedDB implementation
_handleQuotaExceededdeletes metadata rows only. The blobs incache_files, which take up the actual space, stay behind with nothing pointing to them, so the quota handling frees almost nothing._IndexedDbIOSinkkeeps the whole download in aList<int>, which on web costs several times the file size.openReadreturns the whole file as one chunk, andexists()andlength()read the entire blob. Writing in chunks and usinggetKeyorcountforexists()would fix most of this._createTransactionmentions relaxed durability and a ~10x speed-up, but it passes an emptyIDBTransactionOptions.close()doesn't close anything, and theonblockedhandlers are empty.- The schema setup lives in both
IndexedDbFileandIndexedDbCacheInfoRepository, and whichever registers first with the shared connection pool decides the upgrade. A single shared schema definition would be safer. cache_store.dartadds akIsWeb ? await file.exists() : file.existsSync()branch. Usingawait file.exists()on every platform is simpler.
Cancellation
- Two callers for the same URL share one download. If A passes a token and B requests the same URL, B joins A's stream, and cancelling A also fails B. If A has no token, B's token is ignored. Cancellation should apply per caller, or the shared download should stop only once every caller has cancelled.
- Cancelling mid-download leaves a partial file under a new UUID that nothing references, so it never gets cleaned up.
- The sink isn't closed when the response stream throws. The old
.pipe(sink)took care of that.
Tests
The IndexedDB code (repository, connection pool, file and sink, about 1,500 lines) has no automated tests. example/lib/test_indexeddb.dart is a manual demo page. Tests run with flutter test --platform chrome could cover it.
The cancellation tests cover the token itself, HttpFileService when cancelled before the request starts, and one WebHelper case. There's nothing yet for cancelling a queued request, cancelling mid-stream, two callers for one URL, passing the token through CacheManager and ImageCacheManager, or cleaning up partial files.
Suggested way forward
Please close this PR and open two new ones, each branched from upstream/main and following the current CONTRIBUTING.md and PR template.
The first would add web persistence to flutter_cache_manager only. Keep file_system.dart exported, and either make IndexedDB opt-in through Config or keep it as the default and document it in the README. It should also fix the quota handling and memory use described above and add browser tests.
The second would add cancellation. Ideally it stays non-breaking by leaving FileService.get as it is and handling the token in WebHelper and CacheManager. If the signature really has to change, plan it as 4.0.0 and release flutter_cache_manager_firebase alongside it in a separate PR. Either way, fix the shared-download behaviour and the partial-file cleanup, and add the missing tests.
Each PR needs its own version bump, a dated CHANGELOG.md entry that describes the change, and a description that matches what the PR does. Happy to help if anything is unclear, and thanks again for the work so far.
✨ What kind of change does this PR introduce? (Bug fix, feature, docs update...)
Feature
Flutter web has no persistent caching. Cache is stored only in memory and lost on page refresh.
🆕 What is the new behavior (if this is a feature change)?
Cache metadata and files stored in IndexedDB. Data persists across page refreshes and browser sessions.
💥 Does this PR introduce a breaking change?
No - Non-breaking change. Works automatically without code changes. SDK 3.6.0+ now required.
🐛 Recommendations for testing
📝 Links to relevant issues/docs
#480
#339
#361
#122
🤔 Checklist before submitting