- Vincent Koc's Slack crawler just picked up the trick steipete's Discrawl pioneered on Discord: publish your workspace archive to a private git repo and every teammate can query it locally without holding a Slack bot token.
TL;DR
slacrawl v0.4.0 (Apr 22, 2026) adds git-backed archive sync: one publisher runs the Slack bot, exports a compressed snapshot of the SQLite mirror to a private git repo, and everyone else gets the full searchable archive via git pull. No per-user Slack OAuth, no bot credential sprawl, auto-refresh built in. It's the same pattern Peter Steinberger's Discrawl popularised for Discord, now on the Slack side.
What's new in 0.4.0
Three changes ship together and compose into the new workflow:
- Git-backed archive sync workflow — publisher serialises the SQLite mirror to compressed JSONL and commits it to a git remote; consumers clone and restore locally.
- Archive report & share freshness views — consumers can see how stale their clone is before trusting a query, which matters when the publisher's cron fell over yesterday.
- Hardened indexed text + read-path indexes — 0.4.0 also tightens FTS5 storage and adds indexes on the columns query paths hit most.
The release arrived the same day as v0.5.0 (which adds a digest command for windowed per-channel summaries), so if you're jumping in fresh, grab the latest.
Why it matters
Every "export Slack history" tool so far has forced a trade: either each teammate installs their own bot (credential sprawl, admin headache), or one central service hosts everything behind auth (yet another thing to run, pay for, and secure). Git-backed sync collapses both. The bot lives with one person or one CI job. The archive lives in the same private repo where your team already has SSO'd access and audit logs. Querying is local SQLite — fast, offline, scriptable, and it composes with every other tool you already point at a .db file.
The model also quietly fixes the free-tier Slack problem: workspaces that can't afford paid history caps suddenly have an append-only history stored in git forever, and they can rewind to any prior commit to see exactly what the archive looked like on a given day. That's hard to get from Slack itself even on paid plans.
There's a second, subtler win: sharing without re-authing. Contractors, new hires, or cross-org collaborators can read the archive via a normal repo permission — no Slack Guest seats, no bot permissions negotiation, no "I'll DM you the export" Dropbox dance.
Technical facts
- Language: Go (1.25+).
- Storage: local SQLite with FTS5 full-text indexes.
- Data modelled: workspaces, channels (public & private), users, messages, thread replies, structured mentions.
- Sync modes: incremental API sync (default), full backfill, desktop-local recovery from macOS Slack artifacts, Socket Mode live tailing via app tokens, scheduled periodic refresh.
- Distribution: Homebrew, Linux packages, or
go buildfrom source. - Share format: compressed JSONL snapshots committed to a git remote; consumers restore into a local SQLite read-only clone.
slacrawl vs Discrawl vs Slack native
| Capability | slacrawl 0.4.0 | Discrawl 0.4.1 | Slack native search |
|---|---|---|---|
| Target chat | Slack | Discord | Slack |
| Local SQLite + FTS5 | ✅ | ✅ | ❌ |
| Git-backed share | ✅ new | ✅ | ❌ |
| Consumers skip bot creds | ✅ | ✅ | n/a |
| Auto-refresh clones | ✅ | ✅ | n/a |
| Semantic search | not yet | optional (OpenAI / Ollama) | limited |
| License | OSS | MIT | proprietary |
Discrawl shipped the pattern first on Discord; slacrawl now closes parity on Slack. Vincent Koc explicitly cites steipete's tool as the blueprint.
Use cases
- Team knowledge base: turn years of Slack threads into a queryable corpus every engineer can
git pulland grep without waiting on Slack's search. - Free-tier preservation: Slack caps history on free plans — a git snapshot keeps every message forever, with commit timestamps as an accidental audit trail.
- Compliance & legal hold: the git log of the archive doubles as a retention record; rewinding a commit shows exactly what the workspace held on any date.
- LLM & RAG grounding: point a local agent at the SQLite file; FTS5 handles keyword recall today, and nothing stops you from building an embedding sidecar table for semantic search.
- Onboarding & contractors: grant repo read access instead of a Slack Guest seat.
- Offline work: clone once, query on a plane.
Limitations & pricing
slacrawl is free and open source. You do bring your own Slack bot/app token (the publisher still needs channels:history, users:read, and private-channel invites) and a private git repo for distribution. Constraints worth knowing:
- No semantic search in 0.4.0 — Discrawl already has it, slacrawl is FTS5-only for now.
- Snapshot freshness is a cron question; the new share freshness view at least makes staleness visible.
- Large workspaces mean large JSONL blobs — no sharding strategy documented in 0.4.0, so very big archives may want git-LFS or manual partitioning.
What's next
The release-after-release cadence (0.4.0 and 0.5.0 the same day) suggests Vincent is iterating fast. Natural next gaps versus Discrawl: semantic search with pluggable embeddings, sharded snapshots for workspaces in the hundreds of thousands of messages, and hybrid live-tail + git-clone for consumers who want freshness without full publisher rights.
Sources: slacrawl releases, slacrawl README, Discrawl by steipete, Vincent Koc's announcement.
