Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,14 @@ Thanks for helping. The project is young, so small, focused contributions are th
sudo apt install python3-gi gir1.2-gtk-4.0 gir1.2-adw-1 # Debian/Ubuntu
make test # headless core tests, no GTK or network needed
make run # run from the source tree
make start # dev loop on a demo project, restarts on every save
```

Flatpak users get the GTK and libadwaita of the GNOME runtime, not yours. Before submitting UI work, run the same dev loop inside that runtime (the version is read from the Flatpak manifest):

```bash
flatpak install --user flathub org.gnome.Platform//51 # once
make start-flatpak
```

## Ground rules
Expand Down
3 changes: 3 additions & 0 deletions DISTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,7 @@ Il percorso seguente "ripercorre i passi" dei progetti studiati nell'ordine in c
| [awesome-hacktoberfest-2026 #22](https://github.com/Piyushhbhutoria/awesome-hacktoberfest-2026/pull/22) | lista Hacktoberfest (62★), sezione Coding for Beginners | 15 ott 2026 | PR aperta; ultimo merge del maintainer a nov 2025 | attendere merge |
| [ollama/ollama #19001](https://github.com/ollama/ollama/pull/19001) | README Community Integrations (183k★), sezione RAG & Knowledge Bases | 14 ott 2026 | PR aperta; 80+ PR README in coda, merge rari | attendere |
| [awesome-context-engineering #88](https://github.com/yzfly/awesome-context-engineering/pull/88) | lista awesome (211★, molto attiva, merge quasi quotidiani), sezione Memory & Compression accanto a LWC e Dory | 16 ott 2026 | PR aperta (EN + CN). La #69 per openapi-llm-wiki era stata chiusa come "vendor-specific catalog": il maintainer vuole metodi o strumenti riutilizzabili, descritti per meccanismo; ringraziato sulla #69 con link alla #88 | attendere merge |
| [PyPI `gknowledge-mcp`](https://pypi.org/project/gknowledge-mcp/) + [registro MCP ufficiale](https://registry.modelcontextprotocol.io/v0/servers?search=gknowledge) | pacchetto e registro MCP (`io.github.openapi/gknowledge`) | 16 ott 2026 | **pubblicati** 0.2.2 dal workflow `mcp.yml` (trusted publishing + OIDC); `uvx gknowledge-mcp --init` verificato | PR ad awesome-mcp-servers; controllare Glama e PulseMCP |
| PyCoder's Weekly | newsletter Python (form Google) | 14 ott 2026 | modulo compilato, **invio bloccato** dal classificatore auto mode | inviare a mano con i testi di `emerging.md` |
| TabNews, r/developersIndia, Hashnode, Viblo, Techpoint Africa, organizzatori Hack Day | comunità e stampa in Africa e paesi emergenti | 14 ott 2026 | testi pronti in `emerging.md` | pubblicare dal proprio account; r/developersIndia solo nel thread Showcase dell'8 nov |

Expand Down Expand Up @@ -468,6 +469,8 @@ Il messaggio per questo pubblico è **offline, senza API a pagamento, su un port
| 13 | Attivare Discussions, annunciarvi le release, chiedere ai primi utenti di seguire il repo (Watch) | noi | XS | 0–2 | trattiene i visitatori, non ne porta |
| — | Homepage del repo verso la pagina delle release o di installazione | noi | XS | — | evita un link fuorviante nelle anteprime |

**Stato al 16 ottobre 2026.** Fatti: 11 (topic: tolti api-marketplace, certified-api, openapi, openclaw, ai, claude-skills; aggiunti gnome, libadwaita, karpathy, ollama, mcp-server, obsidian; limite di 20) e homepage (ora `openapi.github.io/gknowledge`). Il 7 è in [PR #41](https://github.com/openapi/gknowledge/pull/41) (merged; pacchetto e registro pubblicati il 16 ott): pacchetto separato `gknowledge-mcp` (solo core e server, senza GTK, con `--init`), `server.json`, `glama.json` e workflow `mcp.yml`. Dopo il merge servono il trusted publisher su PyPI e l'environment `pypi`; poi la PR ad awesome-mcp-servers. Da fare: 9, 10, 13.

**Sequenza.** Settimane 1–2: preparazione (7, 9, 10, 11, 13 e homepage da parte nostra; pin, outreach pronta, karma Reddit, 3–5 commenti HN veri e il video da parte dell'utente). Settimana 3: tutto il lancio in 48 ore (Show HN, social, LinkedIn e blog il primo giorno; r/LocalLLaMA e r/ObsidianMD il secondo; r/selfhosted il venerdì), obiettivo 30+ stelle in 24 ore. Settimane 4–5: v0.3 con una funzione dal nome riconoscibile, TWIG, Phoronix, r/gnome, r/linux. Settimane 6–8: risposte nei thread, Medium, thread Showcase di r/developersIndia dell'8 novembre, controllo delle PR aperte.

**Stima.** Con tutto il piano, la somma dei valori centrali è circa **+80–110 stelle**. Senza Show HN scende a **+45–70**: le 100 in 8 settimane diventano improbabili senza HN o un giorno in trending. L'alternativa a puntare tutto su HN è la combinazione Reddit + 48 ore concentrate.
Expand Down
22 changes: 21 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,14 @@ METAINFODIR := $(DATADIR)/metainfo

INSTALL := install

.PHONY: all help check test run dev start demo-svg install uninstall clean dist pypi flatpak
.PHONY: all help check test run dev start start-flatpak demo-svg install uninstall clean dist pypi flatpak

all: help

help:
@echo "gknowledge — targets:"
@echo " make start Dev loop: run on a demo project and auto-restart on source changes"
@echo " make start-flatpak Same loop, but inside the GNOME runtime Flatpak users get"
@echo " make run Run the app from the source tree (opens your last project)"
@echo " make test Run the headless core test suite"
@echo " make check Byte-compile all sources (syntax check)"
Expand Down Expand Up @@ -63,6 +64,25 @@ start:
fi
XDG_CONFIG_HOME=$(CURDIR)/$(SANDBOX)/config $(PYTHON) build-aux/dev/watch.py $(DEMO)

# The same loop inside the Flatpak runtime: the host GTK and libadwaita can
# differ from what Flatpak users run, so check UI work here before a release.
# The runtime version is read from the manifest to keep the two in sync.
RUNTIME := org.gnome.Platform//$(shell sed -n "s/^runtime-version: *'\(.*\)'/\1/p" build-aux/flatpak/$(APP_ID).yaml)

start-flatpak:
@flatpak info --user $(RUNTIME) >/dev/null 2>&1 || flatpak info $(RUNTIME) >/dev/null 2>&1 || { \
echo ">> Missing runtime. Install it with:"; \
echo " flatpak install --user flathub $(RUNTIME)"; exit 1; }
@mkdir -p $(SANDBOX)/config
@if [ ! -f $(DEMO)/gknowledge.json ]; then \
echo ">> Creating and ingesting demo project in $(DEMO)"; \
$(PYTHON) build-aux/dev/seed_demo.py $(DEMO); \
fi
flatpak run --share=network --share=ipc --socket=wayland --socket=fallback-x11 \
--device=dri --own-name=$(APP_ID) --filesystem=$(CURDIR) --cwd=$(CURDIR) \
--env=XDG_CONFIG_HOME=$(CURDIR)/$(SANDBOX)/config \
--command=python3 $(RUNTIME) build-aux/dev/watch.py $(DEMO)

install:
@echo ">> Installing gknowledge into $(DESTDIR)$(PREFIX)"
$(INSTALL) -d $(DESTDIR)$(LIBDIR)
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
<img alt="Karpathy LLM Wiki pattern" src="https://img.shields.io/badge/pattern-Karpathy%20LLM%20Wiki-f59e0b?style=flat-square">
<img alt="Sign in with ChatGPT" src="https://img.shields.io/badge/Sign%20in%20with-ChatGPT-10a37f?style=flat-square&logo=openai&logoColor=white">
<img alt="Ollama" src="https://img.shields.io/badge/Ollama-supported-000000?style=flat-square">
<img alt="Flatpak planned" src="https://img.shields.io/badge/Flatpak-planned-4a86cf?style=flat-square&logo=flatpak&logoColor=white">
<img alt="Flatpak" src="https://img.shields.io/badge/Flatpak-available-4a86cf?style=flat-square&logo=flatpak&logoColor=white">
<img alt="PRs welcome" src="https://img.shields.io/badge/PRs-welcome-ff69b4?style=flat-square">
</p>

Expand Down Expand Up @@ -179,7 +179,7 @@ Nothing is sent anywhere unless you pick a provider that is not offline.
| Native GNOME / GTK 4 | **yes** | Tauri | Tauri | Electron | Electron | web + Electron |
| Works offline, no key | **yes** | needs a model | AI optional | agent-driven | local model | self-host |
| Plain Markdown vault | **yes** | yes | yes | yes | yes | reads yours |
| Linux packaging | `.deb`, Flatpak planned | `.deb`, AppImage | AppImage, `.deb` | `.deb`, `.rpm` | AppImage | pip, Docker |
| Linux packaging | `.deb`, Flatpak | `.deb`, AppImage | AppImage, `.deb` | `.deb`, `.rpm` | AppImage | pip, Docker |
| License | MIT | GPL-3.0 | AGPL-3.0 | GPL-3.0 | AGPL-3.0 | AGPL-3.0 |

Details and history of each project are in [`DISTRIBUTING.md`](DISTRIBUTING.md).
Expand Down
39 changes: 35 additions & 4 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,37 @@

Actionable backlog derived from [`ROADMAP.md`](ROADMAP.md). Ordered by what unblocks the first 100 stars. `[ ]` open, `[x]` done.

## Feedback from dev.to: Flatpak first, drift over hallucination (2026-10-17)

Source: [comment by Justin (@fromthearchitect)](https://dev.to/fromthearchitect/comment/3i4d5) on the launch article. Data/AI architect on Fedora, writing a GNOME photo app in Rust; starred the repo and will try gknowledge "once a Flatpak is available". Follow-up is tracked in `DISTRIBUTING.md` §8.1 (dev.to row).

Flatpak (the `.deb` and `install.sh` do nothing for Fedora, Arch or openSUSE users)
- [x] Flatpak build is green in CI (`appstreamcli compose` succeeds since the GNOME 51 bump); every release attaches a bundle and `repos.yml` publishes the remote at `openapi.github.io/gknowledge` (closes #24).
- [x] `flatpak install --user https://openapi.github.io/gknowledge/io.gknowledge.Gknowledge.flatpakref` works on Ubuntu (2026-10-19): pulls GNOME 51 from Flathub, installs 0.2.2, the app launches.
- [ ] Inside the Flatpak: create a project, import through the portal, ingest, reopen after a restart.
- [ ] Verify on a non-Ubuntu distro (Fedora VM or toolbox) before replying to Justin.
- [x] README: badge and comparison table no longer say "Flatpak planned" (install section left as is).
- [x] Vendor `pypdf` 6.19.0 in the manifest (the 0.2.2 bundle has `No module named 'pypdf'`, so PDF import fails). Builds green on PR #42.
- [ ] Push a release so the published bundle includes the stale-source check, then tag the remote as tested.
- [ ] Make the CI and release Flatpak jobs blocking (drop `continue-on-error`) now that they pass.
- [ ] Flathub itself stays paused (AI-assisted code policy, see P0 below); say so plainly if asked, and point to the Pages remote.

Develop against the runtime users get
- [x] `make start-flatpak`: the `make start` loop run inside `org.gnome.Platform` (version read from the manifest), so host GTK/libadwaita differences show up before release.
- [x] CONTRIBUTING documents `make start-flatpak`.
- [ ] With Justin's permission, link his GNOME SDK setup write-up from CONTRIBUTING.

Drift, not hallucination
- [x] `core/lint.py`: `ChangeDetector` flags ingested sources whose original file no longer matches the ingested hash, with the pages that cite them.
- [x] `IngestionManager.refresh_source()`: copy the new version, update hash and size, re-ingest (builds on #30's `reingest_source`).
- [x] UI: changed sources show "changed since ingest" with a warning icon; the Re-ingest button becomes "Update"; a toast on project open and when the window regains focus.
- [x] MCP tool `stale_sources` so agents can run the lint step of Karpathy's pattern.
- [x] Design written up in PR #42, crediting the suggestion; invite Justin to review it when replying.
- [ ] Mark affected pages as possibly outdated in the Knowledge pane (badge on the page, note in the page header).
- [ ] Trigger the check from `Gio.FileMonitor` on origin files (#3) instead of only on open and focus.
- [ ] Sources imported through the document portal: confirm `origin_path` stays readable across sessions inside the Flatpak.
- [ ] Emit drift as review items once the Review queue is split (#17).

## UX review with a real ChatGPT plan (2026-10-13)

Test: 6 short AI-paper notes (about 300 words each), provider "ChatGPT plan", model picked automatically (gpt-6.1-sol).
Expand Down Expand Up @@ -58,7 +89,7 @@ Speed and feedback
- [ ] Decide the app ID (`io.github.<owner>.Gknowledge` or a verified domain). `io.gknowledge.*` needs a domain we control.
- [x] Bump the runtime to GNOME 51 (current stable on Flathub). Not yet built in CI.
- [x] Replace `--filesystem=home` with FileChooser / Documents portals; keep only `--share=network`. Done in the manifest, untested in a real Flatpak build.
- [ ] Build from `git`/`archive` sources instead of `type: dir`; vendor `pypdf`.
- [ ] Build from `git`/`archive` sources instead of `type: dir` (`pypdf` vendored).
- [ ] Complete MetaInfo: screenshots (URLs still missing; developer, urls, branding added), `developer`, `url`, branding colors, release notes, summary of 35 characters or fewer.
- [ ] Make `flatpak-builder-lint` pass with no exceptions.
- [ ] Add a `.doap` file. (CI workflows for tests, metadata validation and Flatpak build exist; Flatpak job is non-blocking until proven.)
Expand Down Expand Up @@ -92,7 +123,7 @@ Speed and feedback
- [ ] "Insights" panel: weakly connected pages, hubs, suggested missing topics.

### Lint
- [ ] `core/lint.py`: orphan pages, duplicate titles, missing sources, stale pages, broken `[[links]]`.
- [ ] `core/lint.py`: orphan pages, duplicate titles, missing sources, broken `[[links]]` (stale sources done, see the dev.to feedback section).
- [ ] Optional LLM contradiction check.
- [ ] Emit lint findings as review items.
- [ ] Append every lint run to `knowledge/log.md` using the parseable `## [date] op | title` prefix.
Expand Down Expand Up @@ -165,10 +196,10 @@ Tasks:
- [x] `build-aux/arch/PKGBUILD` ready for AUR (needs a tag and `updpkgsums`).
- [x] `build-aux/snap/snapcraft.yaml` draft (untested).
- [x] First release `v0.1.0` (2026-10-13) with `.deb`, version-independent `gknowledge_all.deb` and `install.sh` (`curl … | bash`).
- [ ] Fix the Flatpak build: `appstreamcli compose` fails with `file-read-error` on the MetaInfo (the CI job hides it with `continue-on-error`).
- [x] Fix the Flatpak build: `appstreamcli compose` succeeds in CI since 2026-10-19 (run 39223882741).
- [ ] Test `sudo apt install ./gknowledge_all.deb` on a clean Ubuntu 24.04 VM.
- [ ] Publish a signed APT repository on GitHub Pages (reprepro or aptly) and document `curl | gpg --dearmor` setup.
- [ ] Publish a Flatpak OSTree repo on GitHub Pages with a `.flatpakrepo` file; document `flatpak remote-add`.
- [x] Publish a Flatpak OSTree repo on GitHub Pages with a `.flatpakrepo` file; document `flatpak remote-add` (README documents the `.flatpakref` install).
- [ ] Register `gknowledge` on the Snap Store and test the snap on Ubuntu 24.04; decide on `home` plug vs portals.
- [ ] Create a Launchpad account and PPA; upload a source package (`debian/` with debhelper).
- [ ] Submit AUR package.
Expand Down
11 changes: 11 additions & 0 deletions build-aux/flatpak/io.gknowledge.Gknowledge.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,17 @@ finish-args:
# which goes through the document portal inside the sandbox.

modules:
# PDF import; pure Python, installed from the wheel without network access.
- name: python3-pypdf
buildsystem: simple
build-commands:
- pip3 install --no-index --no-deps --no-build-isolation --find-links="file://${PWD}"
--prefix=${FLATPAK_DEST} pypdf
sources:
- type: file
url: https://files.pythonhosted.org/packages/3c/2c/c43c03eaf630435f023f1dc61ec4a4a78951ad5530a62c71cc89bde307b7/pypdf-6.19.0-py3-none-any.whl
sha256: 7e5d6e730e7dae87d560a2cee218b852f6498c8be61966f3cd02ead971e48d14

- name: gknowledge
buildsystem: simple
build-commands:
Expand Down
32 changes: 32 additions & 0 deletions gknowledge/core/ingestion.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
from .extraction import ExtractionPipeline
from .hashing import hash_file
from .knowledge import KnowledgeWriter
from .lint import origin_of
from .models import Job, JobState, PageKind, Source, SourceStatus, new_id, now
from .project import Project

Expand Down Expand Up @@ -126,6 +127,37 @@ def reingest_source(self, source_id: str) -> Source | None:
writer.append_log("re-ingest", source.title)
return source

def refresh_source(self, source_id: str) -> Source | None:
"""Take the current content of a source's original file and re-ingest it.

Used when lint reports the original changed after ingestion: the stored
copy is replaced, the hash updated, and the old knowledge detached.
"""
if self.is_running():
raise RuntimeError("Cannot refresh a source while ingestion is running.")
source = self.store.get_source(source_id)
if source is None:
return None
origin = origin_of(source)
if origin is None:
raise FileNotFoundError(f"Original file is missing: {source.origin_path}")
digest = hash_file(origin)
other = self.store.source_by_hash(digest)
if other and other.id != source.id:
raise ValueError(f"Same content is already imported as “{other.title}”.")

old_path = self._managed_path(self.project.sources_dir, source.rel_path)
dest = self.project.sources_dir / f"{origin.stem}-{digest[:8]}{origin.suffix}"
shutil.copy2(origin, dest)
if old_path is not None and old_path.is_file() and old_path != dest.resolve():
old_path.unlink()
source.rel_path = dest.name
source.sha256 = digest
source.size = dest.stat().st_size
self.store.upsert_source(source)
self.store.log("source.refresh", source.title)
return self.reingest_source(source.id)

def remove_source(self, source_id: str) -> Source | None:
"""Remove the source and clean up knowledge no longer backed by it."""
if self.is_running():
Expand Down
74 changes: 74 additions & 0 deletions gknowledge/core/lint.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
"""Wiki lint: find knowledge that no longer matches its sources.

A generated wiki rarely goes wrong because the model invented something; it
goes wrong because the source moved on and the page did not. Every source
keeps the SHA-256 of the file it was ingested from and every page keeps the
ids of the sources behind it, so drift is a hash comparison, not a guess.

Only the stale-source check lives here for now; orphan pages, broken
``[[links]]`` and duplicate titles belong in this module too.
"""

from __future__ import annotations

import os
from dataclasses import dataclass, field
from pathlib import Path

from .hashing import hash_file
from .models import Page, Source, SourceStatus
from .project import Project


@dataclass
class StaleSource:
source: Source
pages: list[Page] = field(default_factory=list) # pages that cite it


class ChangeDetector:
"""Compare each source's original file with the content that was ingested.

Hashes are memoised by (path, size, mtime), so calling :meth:`stale_sources`
on every refresh only re-reads files that were touched since the last call.
"""

def __init__(self, project: Project):
self.project = project
self._memo: dict[tuple[str, int, int], str] = {}

def current_hash(self, source: Source) -> str | None:
"""Hash of the original file now, or None if it is gone or unreadable."""
if not source.origin_path:
return None
try:
st = os.stat(source.origin_path)
except OSError:
return None
key = (source.origin_path, st.st_size, st.st_mtime_ns)
if key not in self._memo:
try:
self._memo[key] = hash_file(source.origin_path)
except OSError:
return None
return self._memo[key]

def is_stale(self, source: Source) -> bool:
if source.status != SourceStatus.INGESTED:
return False # nothing generated from it yet
digest = self.current_hash(source)
return digest is not None and digest != source.sha256

def stale_sources(self) -> list[StaleSource]:
store = self.project.store
return [StaleSource(s, store.pages_for_source(s.id))
for s in store.list_sources() if self.is_stale(s)]


def stale_sources(project: Project) -> list[StaleSource]:
return ChangeDetector(project).stale_sources()


def origin_of(source: Source) -> Path | None:
path = Path(source.origin_path) if source.origin_path else None
return path if path and path.is_file() else None
Loading
Loading