Compare commits
16 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 357049aa7b | |||
| d85ef5621c | |||
| faac180212 | |||
| 76d8a45f37 | |||
| d68d322df6 | |||
| 113c951028 | |||
| 2a4ab676ec | |||
| c0a04f505e | |||
| a63006bb07 | |||
| f8e678215d | |||
| f745f8ec1e | |||
| 33502a6d99 | |||
| 1306ee9518 | |||
| e37546cab1 | |||
| 88c3233fd1 | |||
| 02f3047080 |
32
CHANGELOG.md
32
CHANGELOG.md
@@ -11,6 +11,38 @@ Sections:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## 0.6.8 – Performance & Cache Stabilization (2026-01-19)
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- Fixed excessive GitHub API requests causing rate limiting
|
||||||
|
- Fixed missing persistence of repository versions and metadata
|
||||||
|
- Fixed background enrichment re-running after Home Assistant restart
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- Repository metadata (versions, README, descriptions) is now fetched once in the background
|
||||||
|
- Cached data is reused and only refreshed on explicit user interaction
|
||||||
|
- Improved separation between startup refresh and on-demand updates
|
||||||
|
|
||||||
|
### Improved
|
||||||
|
- Significantly reduced Home Assistant startup time
|
||||||
|
- Greatly reduced GitHub API usage
|
||||||
|
- More reliable version selection for GitHub and HACS repositories
|
||||||
|
|
||||||
|
### Notes
|
||||||
|
- Background tasks may still appear in DEBUG logs (expected behavior)
|
||||||
|
- No functional UI changes in this release
|
||||||
|
|
||||||
|
## [0.6.7] - 2026-01-19
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- Repository metadata loaded on demand is now persisted and restored after Home Assistant restart.
|
||||||
|
- Background enrichment reliably updates latest versions and descriptions for HACS repositories.
|
||||||
|
- Cached repository data is correctly restored on startup for non-installed repositories.
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- Repository details are only considered fully enriched once latest version information is available.
|
||||||
|
- Improved reliability of background cache synchronization without impacting startup performance.
|
||||||
|
|
||||||
## [0.6.6] - 2026-01-18
|
## [0.6.6] - 2026-01-18
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|||||||
51
README.md
51
README.md
@@ -1,3 +1,50 @@
|
|||||||
# bahmcloud_store
|
# Bahmcloud Store (BCS) for Home Assistant
|
||||||
|
|
||||||
Bahmcloud Store for installing costum_components to Homeassistant
|
Bahmcloud Store (BCS) is a provider-neutral store for Home Assistant custom integrations.
|
||||||
|
It allows you to browse, install, update, downgrade, uninstall and restore integrations
|
||||||
|
directly from the Home Assistant UI.
|
||||||
|
|
||||||
|
This README is intended for **end users**.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
### Option A: Home Assistant OS / Supervised (Add-on – recommended)
|
||||||
|
|
||||||
|
1. Open **Settings → Add-ons → Add-on Store**
|
||||||
|
2. Add the Bahmcloud Add-on repository
|
||||||
|
`https://git.bahmcloud.de/bahmcloud/addons`
|
||||||
|
3. Install **Bahmcloud Store Installer**
|
||||||
|
4. Start the add-on
|
||||||
|
5. Restart Home Assistant
|
||||||
|
|
||||||
|
Installed to:
|
||||||
|
/config/custom_components/bahmcloud_store
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Option B: Manual Installation (Container / Core)
|
||||||
|
|
||||||
|
1. Download the latest release
|
||||||
|
2. Copy `custom_components/bahmcloud_store` to:
|
||||||
|
/config/custom_components/bahmcloud_store
|
||||||
|
3. Restart Home Assistant
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Repository Sources
|
||||||
|
|
||||||
|
- **BCS Official**
|
||||||
|
- **HACS**
|
||||||
|
- **Custom**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
Developer documentation:
|
||||||
|
https://git.bahmcloud.de/bahmcloud/bahmcloud_store/src/branch/main/README_DEVELOPER.md
|
||||||
|
|
||||||
|
Full technical documentation:
|
||||||
|
https://git.bahmcloud.de/bahmcloud/bahmcloud_store/src/branch/main/README_FULL.md
|
||||||
|
|||||||
115
README_DEVELOPER.md
Normal file
115
README_DEVELOPER.md
Normal file
@@ -0,0 +1,115 @@
|
|||||||
|
# Bahmcloud Store – Developer Documentation
|
||||||
|
|
||||||
|
For contributors and maintainers.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
Repositories:
|
||||||
|
1) Installer Add-on (HAOS/Supervised) ```https://git.bahmcloud.de/bahmcloud/addons```
|
||||||
|
2) Core Integration ```https://git.bahmcloud.de/bahmcloud/bahmcloud_store```
|
||||||
|
3) Store Index (`store.yaml`) ```https://git.bahmcloud.de/bahmcloud/ha_store```
|
||||||
|
|
||||||
|
### Integration Layout
|
||||||
|
|
||||||
|
custom_components/bahmcloud_store/
|
||||||
|
- __init__.py: setup, panel registration, schedule background after HA started
|
||||||
|
- core.py: index merge, enrichment, install/update/uninstall, backups, restore, caching
|
||||||
|
- providers.py: GitHub/GitLab/Gitea repo info + latest version helpers
|
||||||
|
- metadata.py: read bcs.yaml / hacs.json / hacs.yaml
|
||||||
|
- storage.py: persistent storage (installed, custom, repo cache, hacs cache)
|
||||||
|
- views.py: HTTP API endpoints
|
||||||
|
- update.py: UpdateEntity implementation
|
||||||
|
- repairs.py: (optional) Repairs flow for restart
|
||||||
|
- panel/: UI (panel.js, styles.css, etc.)
|
||||||
|
- manifest.json
|
||||||
|
|
||||||
|
## Runtime Model
|
||||||
|
|
||||||
|
- RepoItem (merged)
|
||||||
|
- Installed repos (storage)
|
||||||
|
- Repo cache (persisted enrichment)
|
||||||
|
- HACS meta cache (mapping owner/repo → name/description)
|
||||||
|
|
||||||
|
## Background
|
||||||
|
|
||||||
|
- Heavy work only after `homeassistant_started`
|
||||||
|
- Refresh: if index unchanged → installed-only refresh + schedule enrichment
|
||||||
|
- Opening a repo triggers `ensure_repo_details()` and persists to cache
|
||||||
|
|
||||||
|
## Providers
|
||||||
|
|
||||||
|
- GitHub: API/releases/tags/atom + raw readme
|
||||||
|
- GitLab: API releases/tags + raw readme
|
||||||
|
- Gitea: API releases/tags + raw readme
|
||||||
|
- Custom: API or HTTPS Request to your Git Provider
|
||||||
|
|
||||||
|
## Metadata
|
||||||
|
|
||||||
|
- Prefer `bcs.yaml`, fallback `hacs.json` / `hacs.yaml`
|
||||||
|
- Populate name/description/category/author/maintainer
|
||||||
|
|
||||||
|
## HTTP API (excerpt)
|
||||||
|
|
||||||
|
Base: /api/bcs
|
||||||
|
|
||||||
|
- GET /api/bcs
|
||||||
|
- POST /api/bcs?action=refresh
|
||||||
|
- GET /api/bcs/readme?repo_id=...
|
||||||
|
- GET /api/bcs/versions?repo_id=...
|
||||||
|
- POST /api/bcs/install?repo_id=...&version=...
|
||||||
|
- POST /api/bcs/update?repo_id=...&version=...
|
||||||
|
- POST /api/bcs/uninstall?repo_id=...
|
||||||
|
- GET /api/bcs/backups?repo_id=...
|
||||||
|
- POST /api/bcs/restore?repo_id=...&backup_id=...
|
||||||
|
- (optional) POST/DELETE custom_repo
|
||||||
|
|
||||||
|
## Update Entities
|
||||||
|
|
||||||
|
- Unique id bcs:<repo_id>
|
||||||
|
- Compare installed ref vs latest ref
|
||||||
|
- Dispatcher signal on refresh/install/update
|
||||||
|
|
||||||
|
## Storage
|
||||||
|
|
||||||
|
- JSON in HA `.storage`
|
||||||
|
- async read/write helpers
|
||||||
|
- repo cache applied on startup
|
||||||
|
|
||||||
|
## Contributing to **BCS Official**
|
||||||
|
|
||||||
|
1) Add pull request to `https://git.bahmcloud.de/bahmcloud/ha_store` (with your integration added to) `store.yaml`)
|
||||||
|
2) Add entry:
|
||||||
|
```yaml
|
||||||
|
- name: Your Integration Name
|
||||||
|
url: https://your-git-hoster.com/your-org/your-repo
|
||||||
|
category: Category (actually only "Integrations" are supported)
|
||||||
|
```
|
||||||
|
3) (Recommended) Add `bcs.yaml` to your repo:
|
||||||
|
```yaml
|
||||||
|
name: Your Integration Name
|
||||||
|
description: One-liner for the store (optional, store information are also catched from git repository)
|
||||||
|
category: Integrations (actually only supported)
|
||||||
|
author: Your Name
|
||||||
|
maintainer: Your Handle
|
||||||
|
```
|
||||||
|
4) Open PR; validation checks: reachable, has `custom_components/<domain>/manifest.json`, sensible metadata
|
||||||
|
5) Merge → appears in **BCS Official** after refresh
|
||||||
|
|
||||||
|
## Coding Guidelines
|
||||||
|
|
||||||
|
- Async I/O, no blocking event loop
|
||||||
|
- Respect provider rate limits
|
||||||
|
- Clean logging around refresh/install/update/restore
|
||||||
|
- Keep UI responsive; throttle updates
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Planed Features
|
||||||
|
|
||||||
|
- Add Downloads and install for category "Dashboard"
|
||||||
|
- Add Downloads and install for category "Template"
|
||||||
|
- Add Downloads and install for category "Theme"
|
||||||
|
- Add Downloads and install for category "Blueprint"
|
||||||
|
-
|
||||||
|
|
||||||
|
|
||||||
217
README_FULL.md
Normal file
217
README_FULL.md
Normal file
@@ -0,0 +1,217 @@
|
|||||||
|
# Bahmcloud Store – Full User Guide
|
||||||
|
|
||||||
|
This guide explains **all features** of Bahmcloud Store (BCS) for Home Assistant.
|
||||||
|
It is written for users and admins who want a complete, practical reference.
|
||||||
|
|
||||||
|
> BCS lets you install & manage custom integrations from **GitHub/GitLab/Gitea** and your own sources, with backups, restore, and version pinning.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Contents
|
||||||
|
- Concepts
|
||||||
|
- Sources (BCS / HACS / Custom)
|
||||||
|
- UI Overview
|
||||||
|
- Finding Integrations
|
||||||
|
- Installing
|
||||||
|
- Selecting Versions / Downgrading
|
||||||
|
- Updating
|
||||||
|
- Uninstalling
|
||||||
|
- Backups & Restore
|
||||||
|
- Custom Repositories
|
||||||
|
- HACS Repositories
|
||||||
|
- Update Entities in Home Assistant
|
||||||
|
- Background Caching & Performance
|
||||||
|
- Restart Required
|
||||||
|
- Troubleshooting
|
||||||
|
- FAQ
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Concepts
|
||||||
|
|
||||||
|
- **Sources**:
|
||||||
|
- **BCS Official** → entries from index (`store.yaml`) made and added from Bahmcloud
|
||||||
|
- **HACS** → official HACS integrations list (toggleable)
|
||||||
|
- **Custom** → manual entries you add locally
|
||||||
|
- **Install location**: `/config/custom_components/<domain>`
|
||||||
|
- **Backup**: BCS keeps pre‑update copies in `/config/.bcs_backups/<domain>/<timestamp>/`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Sources (BCS / HACS / Custom)
|
||||||
|
|
||||||
|
Each repository card shows a **source badge**:
|
||||||
|
- **BCS Official** – from Bahmcloud index
|
||||||
|
- **HACS** – from HACS official list (enable with the toggle)
|
||||||
|
- **Custom** – added by you
|
||||||
|
|
||||||
|
You can **filter by source** with the **Source** dropdown (All / BCS Official / HACS / Custom).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## UI Overview
|
||||||
|
|
||||||
|
Top bar:
|
||||||
|
- **Search** (name/description)
|
||||||
|
- **Source** filter (All/BCS Official/HACS/Custom)
|
||||||
|
- **Category** filter (actually only Integrations are supported)
|
||||||
|
- **State-Filter** filter (All/Installed/Not installed/Update available/Custom repos)
|
||||||
|
- **Sort** (name (A-Z)/Updates first/Installed first)
|
||||||
|
- **HACS official** toggle (on/off)
|
||||||
|
|
||||||
|
Repository card:
|
||||||
|
- Name, description, badges (source, installed/update), category
|
||||||
|
- Buttons: **Install / Update / Uninstall**
|
||||||
|
- **Readme** expandable
|
||||||
|
- **Open** to see details (available versions, metadata)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Finding Integrations
|
||||||
|
|
||||||
|
1. Use **Search** to filter by keywords.
|
||||||
|
2. Combine with **Category** and **Source**.
|
||||||
|
3. Sort to surface desired results.
|
||||||
|
|
||||||
|
Descriptions and latest versions are filled progressively by a background process; opening a repo loads details on demand.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Installing
|
||||||
|
|
||||||
|
1. Open a repository.
|
||||||
|
2. Optionally select **Install version** (default: **Latest**).
|
||||||
|
3. Click **Install** and wait for confirmation.
|
||||||
|
4. Follow the **Restart required** prompt. (or use restart toggle in HomeAssistant Settings)
|
||||||
|
|
||||||
|
**What happens internally**
|
||||||
|
- BCS downloads the repository ZIP for the selected version (release/tag/branch).
|
||||||
|
- It extracts all integrations found under `custom_components/<domain>` and deploys them.
|
||||||
|
- It saves the **installed version (ref)** to track updates reliably, even if the repo’s own `manifest.json` is wrong/outdated.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Selecting Versions / Downgrading
|
||||||
|
|
||||||
|
- Use the **Install version** dropdown in the detail view.
|
||||||
|
- Choose **Latest** or a previous **release/tag**.
|
||||||
|
- Installing a chosen ref **pins** the integration to that ref (no surprise updates).
|
||||||
|
- You can upgrade again later by selecting **Latest** and clicking **Update**.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Updating
|
||||||
|
|
||||||
|
- The **Update** button appears when `latest_version` differs from your **installed version (ref)**.
|
||||||
|
- Updates are also available via **Home Assistant → Settings → Updates** (native Update entity).
|
||||||
|
- Clicking **Update** runs the same safe pipeline as **Install** (with backup).
|
||||||
|
|
||||||
|
**Tip:** Opening a repository detail view forces an immediate check for the latest version for that repo.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Uninstalling
|
||||||
|
|
||||||
|
- Click **Uninstall** on the repository.
|
||||||
|
- BCS removes the integration folders under `custom_components/<domain>`.
|
||||||
|
- The installed state is cleared in the Store.
|
||||||
|
- Restart Home Assistant if prompted.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Backups & Restore
|
||||||
|
|
||||||
|
Before an update/install over existing files, BCS creates a backup:
|
||||||
|
|
||||||
|
```
|
||||||
|
/config/.bcs_backups/<domain>/<timestamp>/
|
||||||
|
```
|
||||||
|
|
||||||
|
**Restore**:
|
||||||
|
1. Open the repository in store.
|
||||||
|
2. Select **Restore…**.
|
||||||
|
3. Pick one of the **last backups** (up to retention limit). (Version details are showed in ())
|
||||||
|
4. Confirm – BCS restores files and reconciles installed version to the restored ref.
|
||||||
|
5. Restart Home Assistant if prompted.
|
||||||
|
|
||||||
|
If the old backup lacks metadata, BCS best‑effort derives the installed version from the backup’s `manifest.json`, or marks the ref as `restored:<timestamp>` so updates remain possible.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Custom Repositories
|
||||||
|
|
||||||
|
You can add any public repository (GitHub/GitLab/Gitea/Any other Git Hoster). BCS will attempt to detect:
|
||||||
|
- provider & default branch
|
||||||
|
- latest version (release/tag/atom)
|
||||||
|
- repo metadata (prefer `bcs.yaml`, fallback `hacs.json/hacs.yaml`)
|
||||||
|
- readme (common filenames)
|
||||||
|
|
||||||
|
**Add an custom repository**(typical flows):
|
||||||
|
- From the Store UI
|
||||||
|
1. If you start up Bahmcloud Store, you are on tab "Store"
|
||||||
|
2. Go to tab "Manage"
|
||||||
|
3. Add Repository URL (Example: ``https://git.bahmcloud.de/bahmcloud/bahmcloud_store``) and (optional) Name you want to show up (will be grabbed from GIT Repository)
|
||||||
|
4. Repository show up in Custom Repositories-list and in "Store" Tab, if available
|
||||||
|
|
||||||
|
**Remove an custom repository**(typical flows):
|
||||||
|
- From the Store UI
|
||||||
|
1. If you start up Bahmcloud Store, you are on tab "Store"
|
||||||
|
2. Go to tab "Manage"
|
||||||
|
3. Your repository you want to remove (if its custom) show up in the list
|
||||||
|
4. Press "Remove" Button to delete from Store (Attention: If you Remove installed Repository, you won´t be able to Uninstall from Store, you have to remove by your own (Delete Folder from ```config/custom_components/*```))
|
||||||
|
|
||||||
|
Custom repos get the **Custom** badge and can be filtered via **Source**.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## HACS Repositories
|
||||||
|
|
||||||
|
Enable the **HACS official** toggle to include official HACS integrations.
|
||||||
|
|
||||||
|
- BCS downloads the HACS integration list and maps **human‑readable names/descriptions** from HACS metadata.
|
||||||
|
- HACS entries are **not** part of your BCS-Official Repositories (avoid duplicate entries).
|
||||||
|
|
||||||
|
With many HACS repos, metadata loads in the background; names/descriptions appear progressively and are cached.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Update Entities in Home Assistant
|
||||||
|
|
||||||
|
BCS exposes update entities for installed repos:
|
||||||
|
- Found under **Settings → Updates**
|
||||||
|
- Clicking **Install** triggers BCS update pipeline
|
||||||
|
- Shows **installed** and **latest** versions (BCS ref logic)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Background Caching & Performance
|
||||||
|
|
||||||
|
- **Fast initial list**: index + local cache only
|
||||||
|
- **Background enrichment**: provider info, latest version, metadata, description, readme (best effort)
|
||||||
|
- **On‑demand**: opening a repo triggers immediate enrichment; data is **persisted** to cache
|
||||||
|
- **Persistent cache**: survives HA restarts; speeds up subsequent runs
|
||||||
|
- **Refresh**: immediately rechecks installed repos and key metadata
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Restart Required
|
||||||
|
|
||||||
|
After install, update, or restore, BCS raises a **Restart required** item in Home Assistant (Repairs). You can restart directly from there.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
- **New release but no update**: Open the repo detail once; ensure it’s a **release/tag** (commits alone don’t change the ref).
|
||||||
|
- **Descriptions/Latest missing**: Wait for background enrichment or open the repo detail (forces enrichment). Cached afterwards.
|
||||||
|
(You also can Install new version by selecting in Version tab)
|
||||||
|
- **Slow startup**: BCS schedules heavy work after HA started. Keep indexes reasonable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## FAQ
|
||||||
|
|
||||||
|
- **Backups path?** `/config/.bcs_backups/<domain>/<timestamp>/`
|
||||||
|
- **Install path?** `/config/custom_components/<domain>`
|
||||||
|
- **Downgrade?** Yes, pick an older version and install.
|
||||||
|
- **Restart needed?** Yes, after install/update/restore.
|
||||||
@@ -38,6 +38,14 @@ HACS_INTEGRATIONS_DATA_URL = "https://data-v2.hacs.xyz/integration/data.json"
|
|||||||
HACS_DEFAULT_CATEGORY = "Integrations"
|
HACS_DEFAULT_CATEGORY = "Integrations"
|
||||||
HACS_CACHE_TTL_SECONDS = 60 * 60 * 24 # 24h
|
HACS_CACHE_TTL_SECONDS = 60 * 60 * 24 # 24h
|
||||||
|
|
||||||
|
# Repo enrichment cache:
|
||||||
|
# - persists across restarts
|
||||||
|
# - keeps UI populated (name/description/latest) without blocking startup
|
||||||
|
REPO_CACHE_TTL_SECONDS = 6 * 60 * 60 # 6h
|
||||||
|
# Release/tag lists change rarely and can be expensive for some providers (e.g. GitHub rate limits).
|
||||||
|
# Cache them longer and refresh only on-demand (when the user opens the version selector).
|
||||||
|
VERSIONS_CACHE_TTL_SECONDS = 7 * 24 * 60 * 60 # 7d
|
||||||
|
|
||||||
|
|
||||||
class BCSError(Exception):
|
class BCSError(Exception):
|
||||||
"""BCS core error."""
|
"""BCS core error."""
|
||||||
@@ -109,6 +117,12 @@ class BCSCore:
|
|||||||
self._hacs_meta: dict[str, dict[str, Any]] = {}
|
self._hacs_meta: dict[str, dict[str, Any]] = {}
|
||||||
self._hacs_meta_lock = asyncio.Lock()
|
self._hacs_meta_lock = asyncio.Lock()
|
||||||
|
|
||||||
|
# Persistent per-repo enrichment cache (name/description/latest). Loaded from storage.
|
||||||
|
self._repo_cache_fetched_at: int = 0
|
||||||
|
self._repo_cache: dict[str, dict[str, Any]] = {}
|
||||||
|
self._repo_cache_lock = asyncio.Lock()
|
||||||
|
self._repo_enrich_task: asyncio.Task | None = None
|
||||||
|
|
||||||
# Phase F2: backups before install/update
|
# Phase F2: backups before install/update
|
||||||
self._backup_root = Path(self.hass.config.path(".bcs_backups"))
|
self._backup_root = Path(self.hass.config.path(".bcs_backups"))
|
||||||
self._backup_keep_per_domain: int = 5
|
self._backup_keep_per_domain: int = 5
|
||||||
@@ -142,6 +156,18 @@ class BCSCore:
|
|||||||
self._hacs_meta_fetched_at = 0
|
self._hacs_meta_fetched_at = 0
|
||||||
self._hacs_meta = {}
|
self._hacs_meta = {}
|
||||||
|
|
||||||
|
# Load persisted per-repo enrichment cache (keeps UI populated after restart).
|
||||||
|
try:
|
||||||
|
rc = await self.storage.get_repo_cache()
|
||||||
|
if isinstance(rc, dict):
|
||||||
|
self._repo_cache_fetched_at = int(rc.get("fetched_at") or 0)
|
||||||
|
repos = rc.get("repos")
|
||||||
|
if isinstance(repos, dict):
|
||||||
|
self._repo_cache = {str(k): (v if isinstance(v, dict) else {}) for k, v in repos.items()}
|
||||||
|
except Exception:
|
||||||
|
self._repo_cache_fetched_at = 0
|
||||||
|
self._repo_cache = {}
|
||||||
|
|
||||||
async def _read_manifest_version_async(self) -> str:
|
async def _read_manifest_version_async(self) -> str:
|
||||||
def _read() -> str:
|
def _read() -> str:
|
||||||
try:
|
try:
|
||||||
@@ -254,7 +280,16 @@ class BCSCore:
|
|||||||
refresh_signature = f"{self.last_index_hash}:{len(custom_repos or [])}:{'h' if hacs_enabled else 'n'}:{len(hacs_repos)}"
|
refresh_signature = f"{self.last_index_hash}:{len(custom_repos or [])}:{'h' if hacs_enabled else 'n'}:{len(hacs_repos)}"
|
||||||
|
|
||||||
if self._last_refresh_signature and refresh_signature == self._last_refresh_signature and self.repos:
|
if self._last_refresh_signature and refresh_signature == self._last_refresh_signature and self.repos:
|
||||||
_LOGGER.debug("BCS refresh skipped (no changes detected)")
|
_LOGGER.debug("BCS refresh fast-path (no repo list changes)")
|
||||||
|
|
||||||
|
# Even if the repo list is unchanged, we still want fresh versions/descriptions
|
||||||
|
# for installed repos and we still want background enrichment to keep the
|
||||||
|
# overview populated.
|
||||||
|
if hacs_enabled and self._hacs_meta_needs_refresh():
|
||||||
|
self.hass.async_create_task(self._refresh_hacs_meta_background())
|
||||||
|
|
||||||
|
await self._enrich_installed_only(self.repos)
|
||||||
|
self._schedule_repo_enrich_background()
|
||||||
return
|
return
|
||||||
|
|
||||||
|
|
||||||
@@ -285,9 +320,15 @@ class BCSCore:
|
|||||||
if self._hacs_meta_needs_refresh():
|
if self._hacs_meta_needs_refresh():
|
||||||
self.hass.async_create_task(self._refresh_hacs_meta_background())
|
self.hass.async_create_task(self._refresh_hacs_meta_background())
|
||||||
|
|
||||||
|
# Apply persisted per-repo enrichment cache (instant UI after restart).
|
||||||
|
self._apply_repo_cache(merged)
|
||||||
|
|
||||||
await self._enrich_installed_only(merged)
|
await self._enrich_installed_only(merged)
|
||||||
self.repos = merged
|
self.repos = merged
|
||||||
|
|
||||||
|
# Enrich remaining repos in the background and persist results (non-blocking).
|
||||||
|
self._schedule_repo_enrich_background()
|
||||||
|
|
||||||
self._last_refresh_signature = refresh_signature
|
self._last_refresh_signature = refresh_signature
|
||||||
|
|
||||||
_LOGGER.info(
|
_LOGGER.info(
|
||||||
@@ -478,6 +519,178 @@ class BCSCore:
|
|||||||
_LOGGER.info("BCS HACS metadata cached: repos=%s", len(self._hacs_meta))
|
_LOGGER.info("BCS HACS metadata cached: repos=%s", len(self._hacs_meta))
|
||||||
self.signal_updated()
|
self.signal_updated()
|
||||||
|
|
||||||
|
# ---------- Persistent per-repo enrichment cache ----------
|
||||||
|
|
||||||
|
def _apply_repo_cache(self, merged: dict[str, RepoItem]) -> None:
|
||||||
|
"""Apply persisted per-repo enrichment cache to RepoItem objects.
|
||||||
|
|
||||||
|
This is a fast, no-I/O operation used to keep the UI populated immediately
|
||||||
|
after a Home Assistant restart.
|
||||||
|
"""
|
||||||
|
if not self._repo_cache:
|
||||||
|
return
|
||||||
|
|
||||||
|
for repo_id, r in merged.items():
|
||||||
|
cached = self._repo_cache.get(str(repo_id))
|
||||||
|
if not isinstance(cached, dict) or not cached:
|
||||||
|
continue
|
||||||
|
|
||||||
|
cached_url = cached.get("url")
|
||||||
|
if isinstance(cached_url, str) and cached_url and r.url and cached_url != r.url:
|
||||||
|
# Repo id reused for different URL; ignore.
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Fill gaps only; never override fresh values from current refresh.
|
||||||
|
if (not r.name or r.name == r.url) and isinstance(cached.get("name"), str) and cached.get("name"):
|
||||||
|
r.name = str(cached.get("name"))
|
||||||
|
|
||||||
|
for attr, key in (
|
||||||
|
("owner", "owner"),
|
||||||
|
("provider_repo_name", "provider_repo_name"),
|
||||||
|
("provider_description", "provider_description"),
|
||||||
|
("default_branch", "default_branch"),
|
||||||
|
("latest_version", "latest_version"),
|
||||||
|
("latest_version_source", "latest_version_source"),
|
||||||
|
("meta_source", "meta_source"),
|
||||||
|
("meta_name", "meta_name"),
|
||||||
|
("meta_description", "meta_description"),
|
||||||
|
("meta_category", "meta_category"),
|
||||||
|
("meta_author", "meta_author"),
|
||||||
|
("meta_maintainer", "meta_maintainer"),
|
||||||
|
):
|
||||||
|
if getattr(r, attr, None):
|
||||||
|
continue
|
||||||
|
v = cached.get(key)
|
||||||
|
if v is None:
|
||||||
|
continue
|
||||||
|
if isinstance(v, str):
|
||||||
|
vv = v.strip()
|
||||||
|
if vv:
|
||||||
|
setattr(r, attr, vv)
|
||||||
|
|
||||||
|
def _repo_needs_enrich(self, repo_id: str, r: RepoItem) -> bool:
|
||||||
|
"""Return True if this repo should be enriched in background."""
|
||||||
|
cached = self._repo_cache.get(str(repo_id)) if isinstance(self._repo_cache, dict) else None
|
||||||
|
ts = 0
|
||||||
|
if isinstance(cached, dict):
|
||||||
|
ts = int(cached.get("ts") or 0)
|
||||||
|
|
||||||
|
# Background enrichment should be a one-time best-effort pass.
|
||||||
|
# If we already attempted it once for this repo, do not keep retrying on every refresh.
|
||||||
|
# On-demand (opening repo details / version selector) can still refresh and persist.
|
||||||
|
if bool(cached.get("bg_done")):
|
||||||
|
return False
|
||||||
|
|
||||||
|
# Missing key fields -> enrich.
|
||||||
|
if not r.latest_version:
|
||||||
|
return True
|
||||||
|
if not (r.meta_description or r.provider_description):
|
||||||
|
return True
|
||||||
|
|
||||||
|
# Stale cache -> enrich.
|
||||||
|
if ts <= 0:
|
||||||
|
return True
|
||||||
|
age = int(time.time()) - ts
|
||||||
|
return age > REPO_CACHE_TTL_SECONDS
|
||||||
|
|
||||||
|
def _update_repo_cache_from_repo(self, repo_id: str, r: RepoItem) -> None:
|
||||||
|
self._repo_cache[str(repo_id)] = {
|
||||||
|
"ts": int(time.time()),
|
||||||
|
"url": r.url,
|
||||||
|
"name": r.name,
|
||||||
|
"owner": r.owner,
|
||||||
|
"provider_repo_name": r.provider_repo_name,
|
||||||
|
"provider_description": r.provider_description,
|
||||||
|
"default_branch": r.default_branch,
|
||||||
|
"latest_version": r.latest_version,
|
||||||
|
"latest_version_source": r.latest_version_source,
|
||||||
|
"meta_source": r.meta_source,
|
||||||
|
"meta_name": r.meta_name,
|
||||||
|
"meta_description": r.meta_description,
|
||||||
|
"meta_category": r.meta_category,
|
||||||
|
"meta_author": r.meta_author,
|
||||||
|
"meta_maintainer": r.meta_maintainer,
|
||||||
|
}
|
||||||
|
|
||||||
|
def _schedule_repo_enrich_background(self) -> None:
|
||||||
|
"""Schedule background enrichment for repo details.
|
||||||
|
|
||||||
|
This is non-blocking and safe to call repeatedly.
|
||||||
|
"""
|
||||||
|
if self._repo_enrich_task and not self._repo_enrich_task.done():
|
||||||
|
return
|
||||||
|
self._repo_enrich_task = self.hass.async_create_task(self._repo_enrich_background())
|
||||||
|
|
||||||
|
async def _repo_enrich_background(self) -> None:
|
||||||
|
# Small delay to avoid competing with critical startup work.
|
||||||
|
await asyncio.sleep(2)
|
||||||
|
|
||||||
|
repo_ids = list(self.repos.keys())
|
||||||
|
updated = 0
|
||||||
|
|
||||||
|
# Keep it conservative to avoid rate limits.
|
||||||
|
sem = asyncio.Semaphore(4)
|
||||||
|
|
||||||
|
async def process_one(repo_id: str) -> None:
|
||||||
|
nonlocal updated
|
||||||
|
r = self.repos.get(repo_id)
|
||||||
|
if not r:
|
||||||
|
return
|
||||||
|
if not self._repo_needs_enrich(repo_id, r):
|
||||||
|
return
|
||||||
|
async with sem:
|
||||||
|
try:
|
||||||
|
await self._enrich_one_repo(r)
|
||||||
|
except Exception:
|
||||||
|
_LOGGER.debug("BCS background enrich failed for %s", repo_id, exc_info=True)
|
||||||
|
# Mark as attempted so we don't keep hammering the provider.
|
||||||
|
async with self._repo_cache_lock:
|
||||||
|
cached = self._repo_cache.setdefault(str(repo_id), {})
|
||||||
|
if isinstance(cached, dict):
|
||||||
|
cached["bg_done"] = True
|
||||||
|
cached["bg_done_ts"] = int(time.time())
|
||||||
|
return
|
||||||
|
|
||||||
|
async with self._repo_cache_lock:
|
||||||
|
self._update_repo_cache_from_repo(repo_id, r)
|
||||||
|
cached = self._repo_cache.setdefault(str(repo_id), {})
|
||||||
|
if isinstance(cached, dict):
|
||||||
|
cached["bg_done"] = True
|
||||||
|
cached["bg_done_ts"] = int(time.time())
|
||||||
|
updated += 1
|
||||||
|
|
||||||
|
# Process sequentially but allow limited concurrency.
|
||||||
|
tasks: list[asyncio.Task] = []
|
||||||
|
for repo_id in repo_ids:
|
||||||
|
tasks.append(self.hass.async_create_task(process_one(repo_id)))
|
||||||
|
|
||||||
|
# Flush in batches to limit memory/connection churn.
|
||||||
|
if len(tasks) >= 25:
|
||||||
|
await asyncio.gather(*tasks, return_exceptions=True)
|
||||||
|
tasks = []
|
||||||
|
if updated:
|
||||||
|
await self._persist_repo_cache()
|
||||||
|
self.signal_updated()
|
||||||
|
await asyncio.sleep(0)
|
||||||
|
|
||||||
|
if tasks:
|
||||||
|
await asyncio.gather(*tasks, return_exceptions=True)
|
||||||
|
|
||||||
|
if updated:
|
||||||
|
await self._persist_repo_cache()
|
||||||
|
self.signal_updated()
|
||||||
|
|
||||||
|
async def _persist_repo_cache(self) -> None:
|
||||||
|
async with self._repo_cache_lock:
|
||||||
|
payload = {
|
||||||
|
"fetched_at": int(time.time()),
|
||||||
|
"repos": self._repo_cache,
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
await self.storage.set_repo_cache(payload)
|
||||||
|
except Exception:
|
||||||
|
_LOGGER.debug("BCS failed to persist repo cache", exc_info=True)
|
||||||
|
|
||||||
async def _enrich_and_resolve(self, merged: dict[str, RepoItem]) -> None:
|
async def _enrich_and_resolve(self, merged: dict[str, RepoItem]) -> None:
|
||||||
sem = asyncio.Semaphore(6)
|
sem = asyncio.Semaphore(6)
|
||||||
|
|
||||||
@@ -577,12 +790,19 @@ class BCSCore:
|
|||||||
if not r:
|
if not r:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
# If we already have a latest_version (or provider_description), consider it enriched.
|
# Consider it enriched only if we already have a latest_version and at least
|
||||||
if r.latest_version or r.provider_description or r.meta_source:
|
# some descriptive info (meta/provider). HACS repos often have meta_source set
|
||||||
|
# early, but still need provider latest_version.
|
||||||
|
if r.latest_version and (r.provider_description or r.meta_description or r.meta_source):
|
||||||
return r
|
return r
|
||||||
|
|
||||||
try:
|
try:
|
||||||
await self._enrich_one_repo(r)
|
await self._enrich_one_repo(r)
|
||||||
|
|
||||||
|
# Persist the newly fetched details so they survive a Home Assistant restart.
|
||||||
|
async with self._repo_cache_lock:
|
||||||
|
self._update_repo_cache_from_repo(repo_id, r)
|
||||||
|
await self._persist_repo_cache()
|
||||||
except Exception:
|
except Exception:
|
||||||
_LOGGER.debug("BCS ensure_repo_details failed for %s", repo_id, exc_info=True)
|
_LOGGER.debug("BCS ensure_repo_details failed for %s", repo_id, exc_info=True)
|
||||||
return r
|
return r
|
||||||
@@ -591,7 +811,43 @@ class BCSCore:
|
|||||||
repo = self.get_repo(repo_id)
|
repo = self.get_repo(repo_id)
|
||||||
if not repo:
|
if not repo:
|
||||||
return []
|
return []
|
||||||
return await fetch_repo_versions(self.hass, repo.url)
|
|
||||||
|
# Prefer cached version lists to avoid hammering provider APIs (notably GitHub unauthenticated
|
||||||
|
# rate limits). We refresh on-demand when the user opens the selector.
|
||||||
|
cached = None
|
||||||
|
cached_ts = 0
|
||||||
|
async with self._repo_cache_lock:
|
||||||
|
cached = self._repo_cache.get(str(repo_id)) if isinstance(self._repo_cache, dict) else None
|
||||||
|
if isinstance(cached, dict):
|
||||||
|
cached_ts = int(cached.get("versions_ts", 0) or 0)
|
||||||
|
|
||||||
|
now = int(time.time())
|
||||||
|
if isinstance(cached, dict) and cached.get("versions") and (now - cached_ts) < VERSIONS_CACHE_TTL_SECONDS:
|
||||||
|
return list(cached.get("versions") or [])
|
||||||
|
|
||||||
|
try:
|
||||||
|
versions = await fetch_repo_versions(self.hass, repo.url)
|
||||||
|
except Exception:
|
||||||
|
versions = []
|
||||||
|
|
||||||
|
# If the provider fetch returned only the basic fallbacks ("Latest" + "Branch") but we have
|
||||||
|
# a previously cached richer list, keep using the cached list.
|
||||||
|
if (
|
||||||
|
isinstance(cached, dict)
|
||||||
|
and cached.get("versions")
|
||||||
|
and len(list(cached.get("versions") or [])) > 2
|
||||||
|
and len(versions) <= 2
|
||||||
|
):
|
||||||
|
return list(cached.get("versions") or [])
|
||||||
|
|
||||||
|
# Persist whatever we got (even if small) to avoid repeated calls when rate-limited.
|
||||||
|
async with self._repo_cache_lock:
|
||||||
|
entry = self._repo_cache.setdefault(str(repo_id), {}) if isinstance(self._repo_cache, dict) else {}
|
||||||
|
if isinstance(entry, dict):
|
||||||
|
entry["versions"] = versions
|
||||||
|
entry["versions_ts"] = now
|
||||||
|
await self._persist_repo_cache()
|
||||||
|
return versions
|
||||||
|
|
||||||
def _add_cache_buster(self, url: str) -> str:
|
def _add_cache_buster(self, url: str) -> str:
|
||||||
parts = urlsplit(url)
|
parts = urlsplit(url)
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
{
|
{
|
||||||
"domain": "bahmcloud_store",
|
"domain": "bahmcloud_store",
|
||||||
"name": "Bahmcloud Store",
|
"name": "Bahmcloud Store",
|
||||||
"version": "0.6.6",
|
"version": "0.6.8",
|
||||||
"documentation": "https://git.bahmcloud.de/bahmcloud/bahmcloud_store",
|
"documentation": "https://git.bahmcloud.de/bahmcloud/bahmcloud_store",
|
||||||
"platforms": ["update"],
|
"platforms": ["update"],
|
||||||
"requirements": [],
|
"requirements": [],
|
||||||
|
|||||||
@@ -38,6 +38,7 @@ class BCSStorage:
|
|||||||
- installed_repos: mapping repo_id -> installed metadata
|
- installed_repos: mapping repo_id -> installed metadata
|
||||||
- settings: persistent user settings (e.g. toggles in the UI)
|
- settings: persistent user settings (e.g. toggles in the UI)
|
||||||
- hacs_cache: cached HACS metadata to improve UX (display names/descriptions)
|
- hacs_cache: cached HACS metadata to improve UX (display names/descriptions)
|
||||||
|
- repo_cache: cached per-repo enrichment (names/descriptions/versions) to keep the UI populated after restart
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def __init__(self, hass: HomeAssistant) -> None:
|
def __init__(self, hass: HomeAssistant) -> None:
|
||||||
@@ -61,8 +62,46 @@ class BCSStorage:
|
|||||||
if "hacs_cache" not in data or not isinstance(data.get("hacs_cache"), dict):
|
if "hacs_cache" not in data or not isinstance(data.get("hacs_cache"), dict):
|
||||||
data["hacs_cache"] = {}
|
data["hacs_cache"] = {}
|
||||||
|
|
||||||
|
if "repo_cache" not in data or not isinstance(data.get("repo_cache"), dict):
|
||||||
|
data["repo_cache"] = {}
|
||||||
|
|
||||||
return data
|
return data
|
||||||
|
|
||||||
|
async def get_repo_cache(self) -> dict[str, Any]:
|
||||||
|
"""Return cached per-repo enrichment data.
|
||||||
|
|
||||||
|
Shape:
|
||||||
|
{
|
||||||
|
"fetched_at": <unix_ts>,
|
||||||
|
"repos": {
|
||||||
|
"<repo_id>": {
|
||||||
|
"ts": <unix_ts>,
|
||||||
|
"url": "...",
|
||||||
|
"name": "...",
|
||||||
|
"provider_description": "...",
|
||||||
|
"meta_name": "...",
|
||||||
|
"meta_description": "...",
|
||||||
|
"meta_category": "...",
|
||||||
|
"meta_source": "...",
|
||||||
|
"latest_version": "...",
|
||||||
|
"latest_version_source": "...",
|
||||||
|
"default_branch": "...",
|
||||||
|
"owner": "...",
|
||||||
|
"provider_repo_name": "..."
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
data = await self._load()
|
||||||
|
cache = data.get("repo_cache", {})
|
||||||
|
return cache if isinstance(cache, dict) else {}
|
||||||
|
|
||||||
|
async def set_repo_cache(self, cache: dict[str, Any]) -> None:
|
||||||
|
"""Persist cached per-repo enrichment data."""
|
||||||
|
data = await self._load()
|
||||||
|
data["repo_cache"] = cache if isinstance(cache, dict) else {}
|
||||||
|
await self._save(data)
|
||||||
|
|
||||||
async def get_hacs_cache(self) -> dict[str, Any]:
|
async def get_hacs_cache(self) -> dict[str, Any]:
|
||||||
"""Return cached HACS metadata.
|
"""Return cached HACS metadata.
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user