Skip to content

Add CodeRabbit plugin - #149

Merged
TimWheeler-SQUP merged 4 commits into
mainfrom
work/tw/PLUG-4779-CodeRabbit
Oct 16, 2026
Merged

TimWheeler-SQUP merged 4 commits into
mainfrom
work/tw/PLUG-4779-CodeRabbit

Conversation

@TimWheeler-SQUP

@TimWheeler-SQUP TimWheeler-SQUP commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

🔌 Plugin overview

  • Plugin name: CodeRabbit
  • Purpose / problem solved: Shows whether teams paying for CodeRabbit are using it and whether its reviews help
    • Seat usage and licenses, including seats held by users with no merged PRs
    • Pull request review times and CodeRabbit's estimated review effort
    • How often CodeRabbit findings are accepted, by severity and category
    • AI Deep Scan security findings
    • MCP usage and the admin audit log
  • Primary audience: engineering leaders, platform teams and security teams
  • Authentication method(s): API key, created by a CodeRabbit organization admin. The CodeRabbit API is Enterprise plan only.

Jira: PLUG-4779

🖼️ Plugin screenshots

Plugin configuration

image

Default dashboards

  • Overview
  • Licenses
  • PRs & Reviews
  • Security
  • User
image image

🧪 Test plan

  • squaredup validate passes: 9 data streams, 1 index definition, 5 dashboards
  • Deployed to a SquaredUp organization and connected to a CodeRabbit Enterprise org with an admin API key
  • Config validation: the Authenticate step passes with a valid key; the seat access step reports a warning for a non-admin key
  • Queried every data stream across the supported timeframes and checked row counts against the CodeRabbit UI
  • Checked all five dashboards render, including the User perspective scoped to an indexed user

⚠️ Known limitations

  • Enterprise only: the CodeRabbit API is not available on other plans
  • Review data is capped at Last quarter

📚 Checklist

  • This PR adds a single plugin only
  • Plugin, datastream and UI naming follow SquaredUp guidelines
  • Logo added
  • One or more dashboards added
  • README added including configuration guidance
  • No secrets or credentials included
  • I agree to the Code of Conduct

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added a CodeRabbit integration with organization, repository, user, pull request, review, license, audit log, MCP usage, and security data.
    • Added dashboards for overview metrics, licenses and seat usage, pull requests and reviews, security findings, and individual users.
    • Added organization and seat-access validation, plus API key setup and configuration guidance.
    • Added filters for repositories, users, timeframes, review status, and security finding state.

TimWheeler-SQUP and others added 3 commits October 16, 2026 09:40
Adds a CodeRabbit data source so teams paying for CodeRabbit can see
whether it is being used and whether its reviews help. It covers seat
usage, pull request review times, how often CodeRabbit findings are
accepted, AI Deep Scan security findings, MCP usage and the admin
audit log, with dashboards for an overview, licenses, PRs and reviews,
security, and a per-user perspective.

Column derivations live in JSON value expressions where the platform
allows it. Scripts remain only for what JSON cannot express: turning
nested findings and MCP servers into rows, trimming the day-granular
API window to an exact 24 hours, filtering security findings to
several repositories when the API accepts only one, and deriving the
PR number, because a url shape's label cannot read computed columns.

Page sizes are deliberately mixed. The metrics endpoints use 1000,
CodeRabbit's default, because at 100 per page paging stopped after
one or two pages without an error and dashboards undercounted. Review
comments uses 250 because each pull request carries its findings and
a page of 1000 exceeded the 6MB response limit over a quarter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CodeRabbit audit logs API only matches the search text against the
actor's name, case-insensitively and on partial or full names. The
field was labelled "Search", which suggested it searched every column.
Rename it, add a tooltip, and correct the README description.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@TimWheeler-SQUP
TimWheeler-SQUP requested a review from a team October 16, 2026 13:46
@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository UI (base), Organization UI (inherited)
  • Review profile: ASSERTIVE
  • Plan: Enterprise
  • Run ID: 0c208a34-f1bb-4301-85cf-a2fca9d33cae
📥 Commits

Reviewing files that changed from the base of the PR and between 711bc4b and 201c777.

📒 Files selected for processing: 1
  • .github/CODEOWNERS

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

📝 Walkthrough

The pull request adds a CodeRabbit analytics plugin with API-key setup, data streams for organization and product metrics, indexed objects, five dashboards, and documentation. It also adds a CODEOWNERS rule for the plugin directory.

Changes: CodeRabbit analytics plugin

Layer / File(s) Summary
Plugin setup and object indexing
.github/CODEOWNERS, plugins/CodeRabbit/v1/metadata.json, ui.json, configValidation.json, custom_types.json, defaultContent/scopes.json, indexDefinitions/default.json, docs/README.md
The plugin defines API-key configuration and validation, Organization, Repository, and User types and scopes, and index mappings. The documentation describes setup, data streams, dashboards, and access conditions.
Organization, seat, and activity data
plugins/CodeRabbit/v1/dataStreams/organizations.json, repositories.json, users.json, licenses.json, auditLogs.json, scripts/repositories.js
The streams configure organization, repository, user, license, and audit-log requests and fields. Repository processing adds organization details from context.
Pull-request and review data
plugins/CodeRabbit/v1/dataStreams/pullRequests.json, reviewComments.json, scripts/pullRequests.js, scripts/reviewComments.js
The streams define filters, pagination, and review metadata. Scripts filter by timeframe and flatten pull-request and comment data into rows.
Security and MCP data
plugins/CodeRabbit/v1/dataStreams/securityFindings.json, mcpUsage.json, scripts/securityFindings.js, scripts/mcpUsage.js, scripts/errorHandling/*
The streams configure security-finding and MCP-usage data. Scripts filter findings, flatten MCP usage, and format API errors, including HTTP 403 responses.
Overview and license dashboards
plugins/CodeRabbit/v1/defaultContent/manifest.json, overview.dash.json, licenses.dash.json
The manifest lists five dashboards. The Overview dashboard presents seats, pull-request activity, review findings, and acceptance rates. The Licenses dashboard presents seat counts, utilization, active users, admin activity, and idle seats.
Pull-request and user dashboards
plugins/CodeRabbit/v1/defaultContent/prsAndReviews.dash.json, user.dash.json
The PRs & Reviews dashboard presents review and merge metrics, acceptance rates, findings, and recent merged pull requests. The User dashboard presents selected-user details, activity, acceptance, findings, and recent merged pull requests.
Security dashboard
plugins/CodeRabbit/v1/defaultContent/security.dash.json
The dashboard presents open-finding counts, severity and CWE charts, affected-repository counts, and a findings table for the selected repository.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 201c7

The PR adds CodeRabbit analytics streams and dashboards. No confirmed defect currently makes merging unsafe; confirm the seat-field API semantics before treating those dashboard totals as authoritative.

### Architecture Summary

Architecture risk: 🔵 Low · up to 201c7

The change affects 1 system.

Changed systems: plugins

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — plugins (service) was modified; 29 changed files map to changed impact.

Before / after behavior

  • observed — Modified behavior in plugins/CodeRabbit/v1/configValidation.json: Adds required organization authentication and optional seat-access validation steps, each with a data stream and success and error messages.
  • observed — Modified behavior in plugins/CodeRabbit/v1/custom_types.json: Adds three custom type entries: Organization with the building icon, Repository with the code-branch icon, and User with the user icon; each entry specifies matching source type and singular/plural labels.
  • observed — Modified behavior in plugins/CodeRabbit/v1/dataStreams/auditLogs.json: Adds the auditLogs data-source configuration, including its GET endpoint, timeframe and search arguments, offset paging, error-message path, search UI, audit-entry metadata, and timeframe support.
  • observed — Modified behavior in plugins/CodeRabbit/v1/dataStreams/licenses.json: Adds the licenses data stream configuration, including its request settings, error path, seat metadata, computed available-seat and utilization values, and none timeframe settings.
Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly identifies the primary change: adding the CodeRabbit plugin.
Description check Passed The description is complete and relevant. It explains the plugin purpose, audience, authentication, dashboards, testing, limitations, screenshots, checklist, and Jira reference.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 7…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 7 files. (1 skipped: 1 unsupported.)

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown

🧩 Plugin PR Summary

📦 Modified Plugins

  • plugins/CodeRabbit/v1

📋 Results

Step Status
Scope & version ✅ Passed
Validation ✅ Passed
Deployment 🚀 Deployed

🔍 Validation Details

✅ coderabbit
{
  "valid": true,
  "pluginName": "coderabbit",
  "pluginType": "hybrid",
  "summary": {
    "Data Streams": 9,
    "Import Definitions": 1,
    "Correlation Rules": 0,
    "UI Configuration": true,
    "Has Icon": true,
    "Has Default Content": true,
    "Config Validation": true,
    "Custom Types": true
  }
}

@clarkd

clarkd commented Oct 16, 2026

Copy link
Copy Markdown
Member

Looks awesome - the OOB dashboards are great!

Comment thread plugins/CodeRabbit/v1/metadata.json
@morgan-evans-24

Copy link
Copy Markdown
Contributor

I tested this plugin to ensure everything works as intended. I went through all the default dashboards and datastreams ensuring they all worked. A few things came up, but were later revealed to be limitations in CodeRabbits API, not this plugin (e.g. certain columns being unexpectedly empty). Everything worked smoothly and robustly. One small change was made after testing in which the parameters for the audit log data stream were made more clear.

@TimWheeler-SQUP
TimWheeler-SQUP merged commit 31334b9 into main Oct 16, 2026
1 check passed
@TimWheeler-SQUP
TimWheeler-SQUP deleted the work/tw/PLUG-4779-CodeRabbit branch October 16, 2026 14:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants