Skip to content

[AI-6672] Polish NiFi README for release#23391

Open
philjlee wants to merge 2 commits intomasterfrom
philip.lee/AI-6672-nifi-readme-polish
Open

[AI-6672] Polish NiFi README for release#23391
philjlee wants to merge 2 commits intomasterfrom
philip.lee/AI-6672-nifi-readme-polish

Conversation

@philjlee
Copy link
Copy Markdown
Contributor

@philjlee philjlee commented Apr 20, 2026

Summary

Pre-release polish of the NiFi Agent integration README (AI-6672). README-only change; no code, assets, or validators touched.

What changed

  • Overview: added a scannable capabilities list (matching the kuma README pattern) and rewrote the lead paragraph so the real differentiator — "no NiFi-side reporting task or JMX exporter" — is up front instead of buried.
  • Minimum Agent version: added the **Minimum Agent version:** line with an HTML-comment TODO placeholder pointing at AI-6674. This PR must not merge to a release cut until AI-6674 fills in the version — an empty Minimum Agent version: line would ship to docs otherwise.
  • Configuration: split into #### Host and #### Containerized (Autodiscovery) subsections. The AD block gives customers a drop-in pod-annotation example and steers credentials to k8s secrets rather than inline.
  • Troubleshooting: previously just "Contact Datadog support." Now covers:
    • Authentication failures (nifi.can_connect = 0): POST /access/token token-flow explanation (verified against nifi/datadog_checks/nifi/api.py:30), required REST API read access on the specific endpoints the check hits, and an identity-provider sanity check.
    • TLS verification: tls_ca_cert for internal CAs; tls_verify: false only as last resort.
    • Missing per-connection / per-processor metrics: reminder they're opt-in and the max_* truncation behavior.
  • Further Reading: linked Apache NiFi REST API docs and System Administrator's Guide.
  • Small prose/style fixes:
    • Relative agent-install link (/account/settings/agent/latest) so it renders on .eu, .gov, and datad0g.com.
    • Normalized "this check" over "this integration" to match kuma convention.
    • Tightened the conf.yaml editing sentence; collapsed the Installation two-liner into one sentence (rendered output unchanged).
    • Backticked default values and literal return values (e.g. `true`, `WARNING`, `1`, `0`).

Scope notes

  • The log-collection README section lives on philip.lee/AI-6669 (not yet merged). Polishing it will be a follow-up PR after AI-6669 lands.
  • No changelog entry required (README-only change; per AGENTS.md, changelogs are required only for Python changes under datadog_checks/).

Review round 1 addressed

  • Dropped unverified NiFi policy names ("view the UI", "access the controller") in favor of the specific REST API endpoints the check reads, with a pointer to the admin guide.
  • Verified POST /access/token claim against the check source before asserting it in user docs.
  • Fixed a non-ASCII em-dash rejected by ddev validate readmes.

Related tickets

  • AI-6672 — this ticket (README)
  • AI-6668 — initial implementation + initial README (merged in #23110)
  • AI-6669 — Logs (in progress, has additional README content)
  • AI-6674 — Pre-release (must fill in the Min Agent version before release)

Test plan

  • `ddev validate readmes nifi` passes
  • Rendered diff self-reviewed on GitHub; Autodiscovery code block renders correctly
  • Verified the `POST /access/token` claim against `nifi/datadog_checks/nifi/api.py:30`
  • Reviewer action: confirm AI-6674 is tracked as a blocker for the release cut so the empty `Minimum Agent version:` line does not ship to public docs.

Pre-release README improvements against master's v0.0.1 baseline:

- Overview: add scannable capabilities list and lead with the real
  differentiator ("no NiFi-side reporting task or JMX exporter").
- Add Minimum Agent version placeholder (to be filled in AI-6674).
- Add Kubernetes/Autodiscovery config example under Configuration.
- Flesh out Troubleshooting: auth failures (with required NiFi
  policies), TLS/self-signed CA handling, and per-connection
  /per-processor cardinality gotchas.
- Add Further Reading section linking upstream NiFi docs.
- Make agent install link relative so it renders on .eu/.gov/datad0g.
- Tighten prose per Elements of Style; normalize "this check" over
  "this integration"; format default values with code.

Log-collection-section polish deferred to a follow-up once AI-6669
merges to master.
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 20, 2026

⚠️ Recommendation: Add qa/skip-qa label

This PR does not modify any files shipped with the agent.

To help streamline the release process, please consider adding the qa/skip-qa label if these changes do not require QA testing.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4d0d04a71e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread nifi/README.md Outdated
The check authenticates to NiFi via `POST /access/token` (JWT bearer token). If authentication fails, the Agent log records an error from the `nifi` check. Common causes:

- Wrong `username` or `password`.
- The NiFi user lacks a policy granting access to the REST API. Grant **view the UI** and **access the controller** at minimum, plus read access on the process groups the check queries.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add the system diagnostics policy to auth guidance

Under a least-privilege NiFi user, these permissions are still insufficient: NifiCheck.check() unconditionally calls _collect_system_diagnostics() before it submits nifi.can_connect=1, and NiFi documents /system-diagnostics as requiring Read - /system (view system diagnostics: https://nifi.apache.org/docs/nifi-docs/rest-api/#system-diagnostics). A user following this guidance with only /flow/process-group access will keep getting 403s and nifi.can_connect=0, so the troubleshooting steps should include the view system diagnostics policy.

Useful? React with 👍 / 👎.

Comment thread nifi/README.md
- Track flow throughput, queued flowfiles, and backpressure across process groups.
- Surface processor status at configurable cardinality, with opt-in per-processor and per-connection metrics.
- Receive NiFi bulletins (errors and warnings) as Datadog events.
- Collect application, user, bootstrap, and request logs.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove unsupported log types from the capability list

This overstates what the released configuration currently ships: a repo-wide search only finds one NiFi log input in nifi/assets/configuration/spec.yaml and the generated conf.yaml.example, and it tails /opt/nifi/logs/nifi-app.log. Users who enable the documented check will not collect user, bootstrap, or request logs from the shipped config, so this bullet should either be limited to the application log or the matching log stanzas should be added before release.

Useful? React with 👍 / 👎.

Replace the bolded "view the UI" and "access the controller"
policy names (flagged by review as unverified) with the concrete
REST API endpoints the check reads. Users still know what to
grant access to, and the NiFi admin guide link in Further Reading
covers policy mechanics.

Also replaces an em-dash with a period (README validator rejects
non-ASCII).
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented Apr 20, 2026

Validation Report

All 20 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and Codecov settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented Apr 20, 2026

Monitor Template Quality Assessment

52 monitors analyzed across 9 integrations.

  • 30 monitors have missing sections
  • Most common missing sections: WHY, IMPACT, HOW_TO_TROUBLESHOOT
Monitors with missing sections
Integration Monitor Missing Sections Suggested Links
kubernetes [Kubernetes] Monitor Kubernetes Deployments Replica Pods IMPACT, WHY, HOW_TO_TROUBLESHOOT, RELATED_LINKS - Logs

Copy link
Copy Markdown
Contributor

@git-thuerk-done git-thuerk-done left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just flagging to be sure to update the TO DO comment but approved

Comment thread nifi/README.md
- Receive NiFi bulletins (errors and warnings) as Datadog events.
- Collect application, user, bootstrap, and request logs.

**Minimum Agent version:** <!-- TODO(AI-6674): set to the first Agent release that ships the NiFi check before public release. -->
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**Minimum Agent version:** <!-- TODO(AI-6674): set to the first Agent release that ships the NiFi check before public release. -->
**Minimum Agent version:** <!-- TODO(AI-6674): set to the first Agent release that ships the NiFi check before public release. -->

Just flagging to be sure to update this link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants