Research & Evaluation
StarlineAI is designed to automate regulatory compliance checks for Indian architectural drawings. Given the fragmented, ambiguous, and jurisdiction-specific nature of building regulations in India, meaningful evaluation requires real-world data, expert review, and context-aware benchmarks.
Dataset Description
- 400+ real-world Indian architectural drawings spanning interiors, residential, mixed-use, and large-scale developments.
- Drawings reflect actual submission standards, drafting practices, and jurisdictional constraints.
Relative depth of regulatory rule coverage across major compliance domains.
Evaluation Methodology
Performance was assessed using a human-in-the-loop evaluation framework, where experienced architects manually reviewed drawings and compared their findings against StarlineAI outputs. Each detected issue was classified as a correct detection, false positive, or missed issue, with human review serving as the reference baseline reflecting real professional compliance workflows.
Cumulative progression of identified compliance issues through the automated detection and human review pipeline.
Performance Metrics
In comparative evaluations, a human reviewer typically identified approximately 80 compliance issues per drawing. StarlineAI identified 75 to 76 issues on average, of which 71 to 72 were confirmed as correct detections after filtering false positives.
Detection Quality
Recall approximately 94 percent, precision between 93 and 95 percent, with a false negative rate of 6 to 7 percent.
Time to Compliance
Automated checks reduce basic compliance review time from approximately 10 minutes to 3–4 minutes per drawing.
Overall Accuracy
90–92%
Observed accuracy in controlled evaluations. Results vary by drawing quality, jurisdiction, and rule complexity.
Failure Modes and Limitations
While StarlineAI demonstrates strong performance, certain regulatory and contextual limitations remain inherent to automated compliance analysis.
Benchmarking Context
There is currently no publicly available global benchmark for automated building code compliance systems. StarlineAI’s evaluation approach emphasizes transparency, real-world data, and India-specific regulatory contexts.