In a clever demonstration of AI's visual recognition capabilities, a developer recently created an automated test that successfully detected white smoke from the Sistine Chapel chimney, signaling the election of a new pope. The test, aptly named a smoke test, showcases how AI can be used for real-time event monitoring through visual assessment rather than traditional image analysis methods.
AI Visual Testing Replaces Traditional Image Analysis
The developer created a test script using an AI testing framework that connects to a live YouTube feed of the Vatican's Sistine Chapel chimney. Rather than implementing complex image processing algorithms to detect smoke color, the system leverages AI's visual recognition capabilities through simple prompts. The test was designed to pass only when white smoke appeared, indicating a successful papal election, and fail if the smoke was black or absent. According to comments from the developer, the test passed right when the smoke started coming out, confirming the election in real-time.
This approach demonstrates how prompt engineering can offer a simpler alternative to traditional computer vision techniques. By instructing the AI to visually assess specific conditions through natural language assertions, developers can avoid the complexity of building custom image analysis solutions.
Test Implementation Details:
- Framework: Custom AI testing framework
- Timeout: 60,000 milliseconds (1 minute)
- AI Model Used: Multiple models tested including Gemini, GPT-4o
- Cost: $0.29 USD for 2 days of monitoring
- Approach: Visual assertion via AI rather than traditional image analysis
- Repository: Available on GitHub (donobu-papal-election-tests)
Cost-Effective Multimodal AI Applications
One particularly interesting aspect of this implementation is its cost-effectiveness. When asked about API expenses for running the test with Google's Flash 2.0, the developer reported spending just $0.29 USD over two days of monitoring. This minimal expenditure highlights how accessible advanced AI capabilities have become, even for specialized use cases like event monitoring.
Several commenters discussed the future potential of such applications, suggesting that on-premises multimodal AI models would make these implementations dramatically better. The developer confirmed they're preparing for this future with a local-first approach including a desktop application, indicating that latency and processing requirements remain considerations for real-time visual assessment tasks.
Instead of AI looking at your code and browser and writing Playwright scripts, AI is directly controlling browser and asserting over tests.
The test serves as both a practical application and a clever play on words. In software development, a smoke test typically refers to preliminary testing to verify basic functionality. Here, the term takes on a literal meaning as the test actually monitors for smoke, creating an amusing technical pun that resonated with the developer community.
While some commenters suggested simpler alternatives—like monitoring news notifications on a phone—the AI-based approach demonstrates how visual recognition can be applied to real-world events with minimal development effort, potentially opening doors for similar applications in other domains requiring visual monitoring and event detection.
Reference: papal_election_smoke.test.ts