# Slack Data Collector

A self-hosted app for **community operators** to offer a **unified invite entry** for their Slack workspace(s) and to track where users came from and which workspace and channel they joined.


## Usage

### For end users

#### Join a workspace

Visit **https://slack.tidb.io/** (or a link shared by the admin), enter your email address, and click **Get Invite**. The page will show one of two actions depending on whether you’re already in the workspace:

- **Already in workspace** → **Enter #everyone channel** (link to the channel).
- **Not in workspace** → **Join #tidb-community workspace** (invite link), with an “Already a member? Visit #channel” link below.

### For administrators

Admin entry: **/admin** (e.g. https://slack.tidb.io/admin). After logging in with your internal Google account, admins can use the following.

#### Dashboard

View comprehensive growth stats over configurable time ranges (up to All Time):
- **Core Metrics**: Track total **Clicks**, **Invites**, and **Joins** with calculated Conversion Rates (CR) and Join Rates (JR).
- **Growth Trends**: Visualize daily, monthly, or yearly trends via interactive ECharts.
- **Referral Ranking**: Identify traffic sources ranked by total link clicks.
- **Popular Channels**: See which Slack channels attract the most invitation requests.

#### Manage teams

To enable the Slack invite page and to keep track of who’s in the Slack workspace and which channels exist, you add the Slack team configuration in the admin (one team per Slack workspace). The configuration includes:

- **Name** — A label for the team (e.g. `tidb-community`).
- **Domain** — The workspace’s Slack domain, i.e. the subdomain in `your-domain.slack.com` (e.g. `tidbcommunity`).
- **Token** — Bot token from a [Slack app](https://api.slack.com/start/apps) with the `users:read.email` scope.
- **Invite link** — Workspace invite URL; see [how to get it in Slack](https://slack.com/help/articles/201330256-invite-new-members-to-your-workspace).

#### Generate invite link

1. Pick workspace, channel, and optional referrer (e.g. `github`, `twitter`).
2. Click **Generate link** and copy the URL.
3. Share or embed the URL (e.g. in articles, event pages).

Example:
```
https://slack.tidb.io/invite?team=tidb-community&channel=everyone&ref=github
```

## Deployment

### Prerequisites

- Docker Compose
- A MySQL or [TiDB Cloud](https://tidbcloud.com/) cluster.
- **Google OAuth Credentials**: Required for `/admin` access.

### Step by step

1. Copy env template and fill values:

    ```bash
    cp .env.example .env
    ```

2. Set required values in `.env`:

    | Variable | Description |
    | :--- | :--- |
    | `DATABASE_URL` | SQLAlchemy connection string (e.g., `mysql+pymysql://...`). |
    | `REDIS_URL` | Redis URL for Celery broker/backend (e.g., `redis://redis:6379/0`). |
    | `GOOGLE_CLIENT_ID` | Client ID from Google Cloud Console for OAuth. |
    | `GOOGLE_CLIENT_SECRET` | Client Secret from Google Cloud Console for OAuth. |
    | `SESSION_SECRET` | A long, random string used to sign session cookies. |
    | `ADMIN_EMAIL_DOMAIN` | Restrict `/admin` access to users with this email domain (e.g., `pingcap.com`). |
    | `BASE_URL` | The public URL of your service (e.g., `https://slack.tidb.io`), used for OAuth redirects. |

3. Start all services:

    ```bash
    docker compose up --build -d
    ```

4. Verify:

    ```bash
    curl "http://localhost:8088/health"
    ```

## Development Guide

### Install dependencies

Install project dependencies and create a virtual environment with [uv](https://docs.astral.sh/uv/):

```bash
uv sync
```

### Run app

Start the FastAPI dev server (reload on file change):

```bash
uv run fastapi dev app/main.py
```

### Run worker

Start the Celery worker to process async tasks (invite, sync, etc.):

```bash
uv run celery -A app.celery_app:celery_app worker -l info
```

For scheduled tasks (e.g. sync), also run beat:

```bash
uv run celery -A app.celery_app:celery_app beat -l info
```

### Run migration

Apply database migrations to bring the schema up to date:

```bash
uv run alembic -c app/alembic.ini upgrade head
```

### Run tests

Run the test suite:

```bash
uv run pytest
```

## FAQs

**Why invite links instead of inviting by API?**
Slack’s programmatic invite API is only available on **Enterprise** plans. We therefore use pre-generated invite links: admins create a link (e.g. from the Generate page or Slack’s “Invite people” flow), save it in the service, and users join the workspace by clicking that link.