Original Post: Learnings from three months of Semgrep Assistant
Three months ago, Semgrep launched the private beta of Semgrep Assistant, an AI tool designed to aid in cybersecurity by automating the review of security alerts. Now available to all Semgrep users, Assistant evaluates security issues and suggests code updates, with results integrated into GitHub and Slack. In the beta phase, the Assistant identified 230 likely false positives, with a 95% user agreement rate. Metrics show it’s effective, with users 1.5 times more likely to fix true positives and 2.2 times more likely to ignore false positives. Improvements in GPT-4 have enhanced these metrics.
Key strategies for advancing metrics include using specific prompts and filtering advice based on the rule category. For autofixes, the tool gauges likelihood of corrections being accurate by asking focused questions, generating confidence ratings to ensure quality. Future directions involve developing Assistant-generated Semgrep rules, improving rule metadata, and creating a feedback loop to enhance triaging and rule writing, ultimately aiming to efficiently manage security backlogs. Users can test these features and provide feedback via community Slack.
Get started with Semgrep Assistant by requesting a demo or enabling it in existing Semgrep Team accounts.
Go here to read the Original Post