Integrate cache, photon-libtcmu and workpool with PhotonLibOS v0.9 upgrade

[liulanzheng]

What this PR does / why we need it:

• Upgrade PhotonLibOS to v0.9.6 and use its maintained cache implementations.
• Integrate photon-libtcmu into OverlayBD. Tcmu-runner is no longer maintained, while photon-libtcmu has accumulated PhotonLibOS adaptations and fixes that increasingly diverge from upstream, making continued synchronization impractical.
• Replace the per-device thread/thread-pool model with a shared Photon WorkPool that distributes TCMU I/O across multiple vCPUs.
• Serialize TCMU completion-ring updates while command execution remains concurrent.
• Improve cross-distribution build compatibility and strengthen direct-I/O file-cache CI coverage.
• Add repeatable TCMU performance testing with regression gates.

For high-concurrency 4 KiB random reads, the WorkPool implementation improves performance by more than 4x compared with the legacy baseline.

Which issue(s) this PR fixes:

N/A

Please check the following list:

• Does the affected code have corresponding tests, e.g. unit test, E2E test?
• Does this change require a documentation update?
• Does this introduce breaking changes that would require an announcement or bumping the major version?
• Do all new files have an appropriate license header?

[Copilot]

Pull request overview

This PR upgrades OverlayBD’s PhotonLibOS dependency to v0.9.6, switches OverlayBD over to Photon-maintained cache implementations, and vendors photon-libtcmu into src/libtcmu (replacing the prior external tcmu dependency). It also introduces a shared Photon WorkPool–based execution model (configurable via workpoolSize) and adds CI tooling for repeatable TCMU performance comparisons and regression gating.

Changes:

• Upgrade PhotonLibOS to v0.9.6 and migrate OverlayBD caching integrations to Photon’s cache APIs/libraries.
• Vendor photon-libtcmu into src/libtcmu and update build + licensing/provenance notices accordingly.
• Add TCMU performance benchmark workflow + scripts; update configuration schema (enableThread → workpoolSize).

Reviewed changes

Copilot reviewed 90 out of 92 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
THIRD_PARTY_NOTICES.md	Adds provenance + licensing notice for bundled photon-libtcmu / tcmu-runner sources.
src/test/simple_credsrv_test.cpp	Updates setsockopt call to templated Photon socket API.
src/test/image_service_test.cpp	Updates setsockopt call and fixes HTTP op lifecycle via destroy_operation.
src/overlaybd/zfile/compressor.cpp	Refactors batch buffer handling to bounded std::array and updates compressor interfaces.
src/overlaybd/stream_convertor/stream_conv.cpp	Updates setsockopt call to templated Photon socket API.
src/overlaybd/registryfs/registryfs.cpp	Switches Base64 + Timeout APIs to Photon utilities (Base64Encode, Timeout::expiration).
src/overlaybd/registryfs/registryfs_v2.cpp	Switches Timeout API to Timeout::expiration.
src/overlaybd/gzip_cache/CMakeLists.txt	Adjusts gzip cache library linkage to include Photon static lib.
src/overlaybd/gzip_cache/cached_fs.h	Introduces gzip cached FS wrapper interface around Photon cache.
src/overlaybd/gzip_cache/cached_fs.cpp	Re-implements gzip cache using Photon’s full-file cache filesystem.
src/overlaybd/gzindex/test/test.cpp	Updates includes and cache construction to Photon cache APIs.
src/overlaybd/gzindex/test/CMakeLists.txt	Updates test linkage to Photon cache link group (photon_cache_lib).
src/overlaybd/CMakeLists.txt	Replaces old cache subdir with gzip_cache and updates library link list.
src/overlaybd/cache/test/random_generator.h	Removes legacy cache test utility (cache subsystem migration).
src/overlaybd/cache/test/CMakeLists.txt	Removes legacy cache test target.
src/overlaybd/cache/pool_store.h	Removes legacy cache pool/store abstraction (superseded by Photon cache).
src/overlaybd/cache/policy/lru.h	Removes legacy cache LRU implementation.
src/overlaybd/cache/ocf_cache/test/flags.conf	Removes legacy OCF cache perf test config.
src/overlaybd/cache/ocf_cache/test/CMakeLists.txt	Removes legacy OCF cache perf test target.
src/overlaybd/cache/ocf_cache/ocf_namespace.h	Removes legacy embedded OCF namespace implementation.
src/overlaybd/cache/ocf_cache/ocf_namespace.cpp	Removes legacy embedded OCF namespace implementation.
src/overlaybd/cache/ocf_cache/ocf_cache.cpp	Removes legacy embedded OCF cache filesystem implementation.
src/overlaybd/cache/ocf_cache/ease_bindings/volume.h	Removes legacy OCF “ease bindings” volume adapter.
src/overlaybd/cache/ocf_cache/ease_bindings/volume.cpp	Removes legacy OCF “ease bindings” volume adapter.
src/overlaybd/cache/ocf_cache/ease_bindings/queue.h	Removes legacy OCF queue adapter.
src/overlaybd/cache/ocf_cache/ease_bindings/queue.cpp	Removes legacy OCF queue adapter.
src/overlaybd/cache/ocf_cache/ease_bindings/provider.h	Removes legacy OCF provider adapter.
src/overlaybd/cache/ocf_cache/ease_bindings/provider.cpp	Removes legacy OCF provider adapter.
src/overlaybd/cache/ocf_cache/ease_bindings/env/utils_mpool.h	Removes legacy OCF env adapter code.
src/overlaybd/cache/ocf_cache/ease_bindings/env/utils_mpool.cpp	Removes legacy OCF env adapter code.
src/overlaybd/cache/ocf_cache/ease_bindings/env/ocf_env.cpp	Removes legacy OCF env adapter code.
src/overlaybd/cache/ocf_cache/ease_bindings/env/ocf_env_list.h	Removes legacy OCF env adapter code.
src/overlaybd/cache/ocf_cache/ease_bindings/env/ocf_env_headers.h	Removes legacy OCF env adapter code.
src/overlaybd/cache/ocf_cache/ease_bindings/ctx.h	Removes legacy OCF ctx adapter.
src/overlaybd/cache/ocf_cache/ease_bindings/ctx.cpp	Removes legacy OCF ctx adapter.
src/overlaybd/cache/ocf_cache/CMakeLists.txt	Removes legacy embedded OCF cache build targets.
src/overlaybd/cache/full_file_cache/CMakeLists.txt	Removes legacy full-file cache build target.
src/overlaybd/cache/full_file_cache/cache_store.h	Removes legacy full-file cache store.
src/overlaybd/cache/full_file_cache/cache_store.cpp	Removes legacy full-file cache store.
src/overlaybd/cache/full_file_cache/cache_pool.h	Removes legacy full-file cache pool.
src/overlaybd/cache/full_file_cache/cache_pool.cpp	Removes legacy full-file cache pool.
src/overlaybd/cache/forwardcfs.h	Removes legacy cached FS forwarding wrappers.
src/overlaybd/cache/download_cache/CMakeLists.txt	Removes legacy download cache build target.
src/overlaybd/cache/CMakeLists.txt	Removes legacy cache umbrella library build.
src/overlaybd/cache/cache.h	Removes legacy cache public API (now Photon cache).
src/overlaybd/cache/cache.cpp	Removes legacy cache implementation.
src/overlaybd/base64.h	Removes local Base64 implementation (now Photon utils).
src/libtcmu/target_core_user_local.h	Adds vendored TCMU UIO ring structures/header.
src/libtcmu/scsi.h	Adds vendored SCSI helper API declarations.
src/libtcmu/scsi_defs.h	Adds vendored SCSI constant/enum definitions.
src/libtcmu/README.md	Documents bundled photon-libtcmu purpose and licensing.
src/libtcmu/LICENSE.Apache2	Adds Apache 2.0 license text for bundled libtcmu distribution choice.
src/libtcmu/libtcmu.h	Adds vendored libtcmu public API.
src/libtcmu/libtcmu_priv.h	Adds vendored libtcmu private structures.
src/libtcmu/libtcmu_common.h	Adds vendored shared libtcmu API/helpers.
src/libtcmu/docs/adr/0001-silent-ignore-unmatched-subtype.md	Adds ADR describing netlink handling for unmatched subtypes.
src/libtcmu/configfs.cpp	Adds configfs interaction implementation for libtcmu.
src/libtcmu/CMakeLists.txt	Adds build definition for vendored libtcmu static library.
src/libtcmu/be_byteshift.h	Adds endian helpers used by libtcmu code.
src/libtcmu/api.cpp	Adds libtcmu helper implementations (CDB/iovec/sense utilities).
src/image_service.h	Updates gzip-cache include path to new location.
src/image_service.cpp	Switches to Photon cache APIs, replaces Base64 decode with Photon utils, updates cache creation calls.
src/exporter_server.h	Updates setsockopt call (but currently contains a compile-breaking macro invocation).
src/example_config/overlaybd.json	Adds workpoolSize configuration key.
src/example_config/overlaybd-registryv2.json	Adds workpoolSize configuration key.
src/config.h	Replaces enableThread config field with workpoolSize.
src/CMakeLists.txt	Adds libtcmu subdirectory and adjusts include dirs for overlaybd-tcmu.
src/api_server.cpp	Updates setsockopt call to templated Photon socket API.
README.md	Documents bundled libtcmu, updates compiler requirement, documents workpoolSize.
CMakeLists.txt	Removes external find_package(tcmu) and ensures e2fs is found; relies on vendored libtcmu.
CMake/Findtcmu.cmake	Removes FetchContent-based external photon-libtcmu dependency.
CMake/Findphoton.cmake	Pins Photon to v0.9.6, forces Photon cache/extfs build options, adds OCF link-group handling.
CMake/Findnl.cmake	Adds libnl discovery module for vendored libtcmu.
CMake/Finde2fs.cmake	Improves e2fsprogs build integration, byproducts, and forces C11 under newer compilers.
.gitmodules	Removes OCF submodule entry (Photon now provides/builds OCF).
.gitignore	Adds .DS_Store.
.github/workflows/tcmu-performance.yml	Adds a workflow to benchmark baseline vs current TCMU implementations.
.github/workflows/release/build.sh	Improves dependency installation robustness (tdnf retry) and pins GCC on older Ubuntu.
.github/scripts/tcmu-performance.sh	Adds scripted fio-based cache warmup + performance measurement with offline verification.
.github/scripts/summarize-tcmu-performance.py	Adds summary + regression gating logic for benchmark results.

Suppressed comments (1)

src/exporter_server.h:62

• Both LOG_ERRNO_RETURN invocations are missing the return-value argument (second parameter), which will not compile and also obscures the intended failure behavior. Use an explicit error return value (e.g., -1) for both bind() and listen() failures.

        tcpserver = photon::net::new_tcp_socket_server();
        tcpserver->setsockopt<int>(SOL_SOCKET, SO_REUSEPORT, 1);
        if (tcpserver->bind(config.exporterConfig().port()) < 0)
            LOG_ERRNO_RETURN(0, , "Failed to bind exporter port `",
                             config.exporterConfig().port());
        if (tcpserver->listen() < 0)
            LOG_ERRNO_RETURN(0, , "Failed to listen exporter port `",
                             config.exporterConfig().port());

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.