Skip to main content

Alpha This is a new service. Help us improve it and give your feedback by email.

Walkthrough Complete - FOI Redaction

You’ve completed the FOI Redaction walkthrough

Walkthrough complete

You've experienced AI-powered PII detection and redaction for FOI requests

Congratulations! In just 10 minutes, you've seen how AI can reduce FOI redaction time by 80% while improving compliance. What would have taken weeks to pilot, you've experienced hands-on right now.

What you've learned

Automatic PII detection

AI read your FOI response and automatically detected names, addresses, phone numbers, email addresses, and National Insurance numbers. No manual line-by-line review required.

Value: 30 minutes of manual review reduced to 15 seconds

95-99% confidence scores

Confidence scores showed 95-99% accuracy on typed documents. You saw how to use thresholds to auto-redact high confidence detections and flag lower scores for human review.

Value: Balanced automation with accuracy - trust the AI on high confidence, review edge cases

Context-aware intelligence

AI understood context - "Manchester" in an address is PII, but "Manchester City Council" is not. It recognized UK postcodes, phone formats, and National Insurance number patterns.

Value: Smart redaction, not just keyword matching - reduces false positives

Compliance protection

Complete audit trail of detected PII, confidence scores, and redaction decisions. Demonstrates due diligence to ICO and reduces risk of accidental personal data disclosure.

Value: GDPR compliance and ICO transparency - provable systematic approach

Committee-ready talking points

Use these points when presenting to decision-makers:

The problem

"FOI officers currently spend 30 minutes per request manually reviewing documents line-by-line to find and redact personal information. With 500 FOI requests per year, that's 250 hours of valuable officer time spent on repetitive redaction work. This creates bottlenecks in meeting the 20-day FOI response deadline and increases risk of missed PII due to review fatigue."

The solution

"AI-powered PII detection automatically scans FOI response documents and identifies names, addresses, phone numbers, emails, and National Insurance numbers in under 30 seconds with 95-99% accuracy. High-confidence detections can be auto-redacted, while edge cases are flagged for quick human review (5-10 seconds per item). This reduces redaction time by 80%."

The value

"Annual time savings of 200 hours - equivalent to 5 weeks of FOI officer capacity - freed for complex exemption decisions and policy work. Cost benefit of £5,000 per year in staff time. Compliance benefit: consistent PII detection reduces risk of accidental disclosure and provides audit trail for ICO transparency. Faster FOI response times improve public service delivery."

The risk

"Low. Pay-per-use pricing means no upfront investment. Trial with 100 FOI requests costs under £50. AI handles heavy lifting (95%+ detection rate) while FOI officers perform final review before publishing responses. System integrates with existing FOI workflows and document management systems. Typical pilot: 3-6 months with 50-100 FOI requests to validate accuracy and ROI."

Next steps

Generate Evidence Pack

Create your business case with what you've learned. Perfect for committee papers.

Generate Evidence Pack

Return to FOI Redaction

Review deployment options, costs, and technical details.

Back to scenario

Try Another

Explore more AI scenarios for local government.

Browse scenarios

Clean up your resources

Good news: Your resources will automatically delete after 2 hours from deployment. However, you can delete them now to stop any further charges immediately.

Warning Deleting your stack will permanently remove all data including:
  • Database content (RDS Aurora)
  • Uploaded files (EFS storage)
  • Any configuration changes you made

Step-by-step deletion

  1. Open the CloudFormation console

    Go to CloudFormation console (US East 1) (opens in new tab)

  2. Find your stack

    Look for a stack named: ndx-try-foi-redaction-[timestamp]

    The timestamp is when you deployed. You can sort by "Created time" to find recent stacks.

  3. Select and delete

    Select the checkbox next to your stack, then click the Delete button.

  4. Confirm deletion

    Click Delete in the confirmation dialog. The stack status will change to DELETE_IN_PROGRESS.

  5. Wait for completion

    Deletion typically takes 5 to 10 minutes. The stack will disappear from the list when complete.

Costs stop after deletion

Once your stack is deleted, you will not incur any further charges for this scenario.
Estimated evaluation cost: Less than $0.50 for a 15-minute trial

Troubleshooting

Stack shows DELETE_FAILED status

This usually happens when resources can't be automatically cleaned up. Common causes:

  • S3 bucket not empty: The bucket may contain files. Go to S3, empty the bucket manually, then retry deletion.
  • Lambda functions in use: Wait a few minutes and retry. Sometimes functions take time to fully stop.
  • Network interfaces still attached: These usually clear within 5 to 10 minutes. Retry the deletion.

To retry deletion: Select the failed stack and click Delete again.

I can't find my stack in the list

If your stack isn't visible:

  • Check the region: Make sure you're viewing US East (N. Virginia) in the console header.
  • Stack already deleted: It may have auto-deleted after 2 hours. No action needed!
  • View deleted stacks: Click "View nested" dropdown and select "Deleted" to see recently deleted stacks.
Extending your evaluation time

If you want to continue testing beyond the 2-hour limit:

  • Resources will auto-delete after 2 hours total (from deployment)
  • Cost: approximately $1-2 per hour of active testing
  • Maximum cost is capped by template configuration

You can redeploy the scenario anytime to start fresh with a new 2-hour window.

Still having trouble? Contact the NDX:Try team or report an issue on GitHub (opens in new tab).

Production considerations

If you're considering deploying this for real FOI request processing:

Integration with existing FOI systems

Production deployment would integrate via:

  • API endpoints to send FOI response documents for PII detection
  • Webhooks to trigger redaction when FOI officers prepare responses
  • Direct integration with FOI case management systems (iCasework, Firmstep, etc.)
  • Queue-based processing for batch redaction of multiple documents
  • Export redacted documents back to FOI system or document management system
Handling low confidence detections

Production systems would:

  • Set confidence threshold (e.g., 95%) for auto-redaction
  • Flag low-confidence detections for FOI officer review
  • Show original document context next to detected PII for easy verification
  • Allow officers to manually mark/unmark PII items
  • Track false positive/negative rates over time to tune confidence thresholds
  • Provide feedback loop to improve detection accuracy for your council's FOI patterns
Data protection and security

Production deployment includes:

  • Encryption at rest for S3 buckets (customer-managed KMS keys)
  • Encryption in transit (TLS 1.2+) for all API calls
  • VPC endpoint for Comprehend (traffic never leaves AWS network)
  • IAM roles with least-privilege permissions
  • CloudTrail logging for audit compliance and ICO requests
  • Automatic document deletion after configurable retention period (e.g., 30 days)
  • GDPR-compliant data processing agreements with AWS
  • UK data residency (all processing in us-east-1 N. Virginia region)
Cost at scale

Real-world cost examples (based on average 3-page FOI response):

  • 500 FOI requests/year: ~£300/year (£0.60 per request)
  • 1,000 FOI requests/year: ~£550/year (£0.55 per request due to volume discounts)
  • 2,000 FOI requests/year: ~£1,000/year (£0.50 per request)

Costs include Comprehend (per text unit), Lambda (per execution), S3 storage, and API Gateway. No upfront licensing or infrastructure costs. Pay only for documents actually processed.

ROI example: For 500 FOI requests/year, £300 cost vs £5,000 staff time savings = £4,700 net benefit.

Handling complex FOI scenarios

Advanced use cases:

  • Partial redaction: Redact PII while preserving organizational context (e.g., redact employee names but keep job titles)
  • Third-party personal data: Detect and redact information about residents, applicants, objectors - not just council employees
  • Multiple exemptions: Combine PII redaction (Section 40) with commercial sensitivity or legal professional privilege redactions
  • Email chain processing: Handle email threads with multiple participants, detecting PII in signatures, headers, and body text
  • Scanned documents: OCR integration for redacting handwritten or poor-quality scanned FOI responses

Questions or feedback?

We'd love to hear about your experience with this walkthrough:

Return to homepage

Need help?

If you're stuck, check the troubleshooting section or contact us.