GitHub expanded its Copilot impact dashboard on November 6, 2026, adding chat and VS Code Agents window engagement. The dashboard now covers nine experiences, while the label “Passive users” becomes “Chat-first users.” For enterprise leaders, the critical point is that the visible activity mix changed, but GitHub says the adoption-phase definitions and cohort sizes did not. Historical comparisons must distinguish a reporting change from an actual behavior change.
What changed—and what did not
GitHub’s November 6 changelog says the enterprise and organization 28-day usage reports now include chat and vscode_agent entries under copilot_feature_engagement.totals_by_feature. Users can appear under multiple features. A user counts as engaged in a feature after using it on at least two days in the 28-day period. The renamed Chat-first group remains No Cohort in the API, and chat and VS Code agent usage do not contribute to adoption phases.
1. Version the metric definition
Every dashboard should carry a metric-definition date. Record that historical reports before the release contain seven feature categories, while later reports can contain nine. Do not silently backfill missing historical categories with zero: zero would imply observed nonuse, whereas the historical feature may simply not have been reported. Label the unavailable values as not collected or not comparable.
2. Never sum overlapping feature users as unique people
Because one developer can be engaged in several features, adding nine feature counts can exceed the number of distinct people. Report feature engagement separately from unique active users. For example, a developer using both chat and the VS Code Agents window belongs to both feature counts. A valid enterprise dashboard should show the deduplication rule and the source population.
3. Preserve the distinction between engagement and adoption phases
GitHub explicitly says the newly reported features do not alter adoption phases. Therefore, a rising chat engagement line does not necessarily indicate movement into a different adoption phase. Present these as separate questions: “Which experiences are people using?” and “Which phase of adoption do they meet?” Mixing the two can mislead executive reviews.
4. Audit cohort labels across exports
The interface now says Chat-first, but the API still returns No Cohort. Update the presentation mapping without rewriting historical raw data. Otherwise, dashboards built by different teams may appear to disagree about cohort size. Document the transformation once and use it consistently in reports, board slides and data pipelines.
5. Check data permissions and policy prerequisites
GitHub says the metrics require the Copilot usage metrics policy to be enabled and appropriate enterprise or organization roles. Verify that an analyst’s empty report is not simply an access or policy configuration problem. Record the reporting scope, export timestamp and permission context. Our analysis of Copilot telemetry gaps explains why incomplete collection can distort adoption estimates.
6. Separate usage from value
More chat interactions do not, by themselves, demonstrate better code quality, faster delivery or lower cost. Build a paired evaluation: feature engagement plus independently measured outcomes such as reviewed change lead time, escaped defects, rework, developer satisfaction and spend per accepted change. Our review metrics analysis shows how a superficially positive rate can differ from verified remediation.
A practical comparison matrix
| Measure | Safe interpretation | Common mistake |
|---|---|---|
| Chat feature engagement | Two or more days in 28-day period | Calling it unique users across all features |
| VS Code agent engagement | Feature-specific participation | Assuming it changes adoption phase |
| Chat-first cohort | Renamed interface label | Treating it as a new cohort in the API |
| Historical seven-feature report | Older reporting coverage | Interpreting absent new categories as zero use |
| Business value | Requires outcome measurement | Equating activity with productivity |
The matrix is an original interpretation of the published metric definitions; it is not a vendor-provided benchmark.
A 30-day reporting correction plan
In week one, snapshot the current schema and document historical availability. In week two, update dashboards and cohort-label mappings. In week three, reconcile feature counts against unique-user totals and review access-policy gaps. In week four, present usage alongside quality and cost outcomes, explicitly separating observed change from instrumentation change. Use the model-routing cost governance checklist to keep model and spending assumptions visible when interpreting developer-tool investment.
Source and limitations
Primary source: GitHub Changelog, “Copilot impact dashboard adds chat and VS Code agent usage,” November 6, 2026. GitHub’s definitions describe reporting behavior; the recommended scorecard and correction plan are independent editorial analysis. They do not assert a measured productivity improvement.
Leave A Comment