The release of smartfunc, a Python library that transforms docstrings into LLM-powered functions, has sparked a vibrant discussion among developers about the optimal approach for integrating large language models into programming workflows. This debate highlights a fundamental question: should LLMs be treated as compilers for high-level specifications or as collaborative assistants in the development process?
The Compiler vs. Assistant Paradigm
The community discussion reveals a strong preference among many developers for treating LLMs as compilers rather than as junior developers. This approach positions LLMs as tools that transform high-level specifications (like docstrings) into functional code, rather than as collaborators that generate code through conversation. One commenter articulated this sentiment clearly, suggesting that treating LLMs like compilers represents a more scalable, extensible and composable mental model than viewing them as development assistants.
However, as Simon Willison (creator of the underlying llm library that powers smartfunc) pointed out, smartfunc doesn't actually function as a compiler. Instead, it converts functions into ones that call an LLM each time they're executed, passing the docstring as a prompt. This clarification sparked further discussion about what a true compiler-like implementation might look like—perhaps generating and caching Python code on first call rather than making repeated API calls.
Runtime Flexibility and Alternative Approaches
A significant limitation highlighted by several developers is the difficulty in using the same function with different LLMs at runtime. The decorator pattern used by smartfunc locks in the model choice at import time, which some find restrictive for production environments. Alternative libraries like think were mentioned as offering more flexibility by explicitly passing LLM objects around.
The discussion also touched on similar implementations across different languages, with developers mentioning comparable tools for JavaScript and interest in Java versions. This cross-language interest demonstrates the growing demand for standardized patterns to integrate LLMs into various development environments.
Similar Libraries Mentioned in Discussion
- smartfunc: Python library that transforms docstrings into LLM functions
- Tanuki: Similar functionality to smartfunc
- promptic: Wraps litellm with support for many model providers
- think: Alternative approach passing LLM objects around for runtime flexibility
- llm-docsmith: Plugin for generating docstrings for existing code
- neuro-lingo: Mentioned as having "pinned" function results
- instructor: More feature-rich alternative with validation support
- marvin: Alternative LLM function library
The Future of Natural Language Programming
Several commenters drew parallels between current LLM integration efforts and historical attempts to make programming more accessible through natural language, such as COBOL in the 1960s. While COBOL represented an early attempt at English-like syntax, modern LLMs potentially bring us closer to Jensen Huang's vision of natural language becoming a programming language.
The discussion highlighted an interesting contrast in workflows as well. While smartfunc uses docstrings to generate functionality, some developers reported using LLMs in the reverse direction—having them generate comprehensive documentation for poorly commented code. This bidirectional relationship between code and natural language hints at evolving development practices where the boundaries between specification, implementation, and documentation become increasingly fluid.
Caching and Local Execution
Looking toward future improvements, developers expressed interest in approaches that would distill LLM functionality into smaller, specialized models for specific functions rather than making repeated API calls to foundation models. This could potentially reduce latency and costs while improving reliability.
The concept of caching known good outputs was also discussed as a way to mitigate the non-deterministic nature of LLMs. Some suggested implementations that would generate code during build steps, verify it against tests or compilation requirements, and then cache successful results—effectively sandboxing what the LLM can alter while maintaining developer control.
As LLM integration patterns continue to evolve, the community seems to be converging on approaches that maintain programmer agency while leveraging the power of AI to handle repetitive or verbose coding tasks. Whether through docstring-powered functions like smartfunc or more sophisticated code generation and caching systems, these tools represent early steps toward a new programming paradigm where natural language and code exist in a more symbiotic relationship.
Reference: smartfunc