Data/Infrastructure, AI

Codium: Code the way you meant it

by Vine Ventures ( 5 min read )
Mar 22, 2023

We’re thrilled to announce Vine is co-leading CodiumAI’s $10.6M Seed alongside TLV Partners and angels from OpenAI, VMWare, and Snyk. 

A year ago, Itamar wrote in a blog about the future of software development, “In the coming years, we will see immense advancements in AI agents that generate programs.” While code-generation LLMs have evolved, CodiumAI has developed a highly proprietary engineering stack and product experience unlocking a new Code Integrity paradigm. As CodiumAI launches into the world this week, we wanted to share parts of the investment memo that got everyone at Vine (between TLV, NYC, and SF) excited about the company. This writing is also meaningful to me because when combing through it with Itamar and Dedy in the midst of a competitive round, it indicated how closely aligned we were on a future where Code Integrity is a category of its own — far beyond existing software testing solutions.

Note: We’ve included in ‘italic’ a re-evaluation of the original thesis described in the memo.

We hope you enjoy the read & welcome your feedback after trying the product!

To: Vine Investment Committee 

From: Dan Povitsky

Subject: IC Memo – June 2022 

Note: Commentary in italic – March 2023


Overview: CodiumAI is an AI Code Integrity platform that leverages code, docstring, and comments to automate software tests and software explainability. The company aims to help developers produce more robust and accurate code, i.e., code faster with confidence. We first met co-founder and CEO, Itamar Friedman, in March 2022. Over the subsequent three months of ideation, we brainstormed with Itamar and co-founder Dedy Kredo, made prospective hire introductions, and hosted numerous diligence calls with industry experts and potential customers. 

Deal Context: We believe our ability to win is rooted in the co-founders’ respect for the depth of our research, the steps we have taken to understand the legitimacy of the market and their vision, and the value we have brought in a short period of time. We plan to submit a term sheet to lead CodiumAI’s $10.6M Seed.

Positive Highlights

Below, we review and largely reinforce the positive highlights which drove our initial thesis to invest in CodiumAI:


  • Software of questionable quality costs companies trillions of dollars. Shifting testing left can significantly alleviate this burden: The cost of fixing software errors compounds as errors move through the SDLC. Preventing software errors is better than fixing them in production, and the best way to prevent errors is to test code often and thoroughly. Reports show that over 80% of software issues lie within the SDLC, with errors in production costing 30x more to fix than in development. We expect developers to seek innovative code testing and integrity tools and believe CodiumAI is well positioned to be that trusted party.

Commentary: Since this memo’s writing, the rise in popularity of AI assistants (i.e., Copilot, ChatGPT, and CodeGPT) has propelled developers to write more code faster. While assistants have delivered productivity benefits, they have also increased the risk of error-filled code slipping into production. One of the reasons for this is the misalignment between developers’ intent and what the LLM assumes before generating code. The misconception that code completion and generation tools are bug-free gives developers false confidence in the accuracy and reliability of their code, increasing the need for code-level testing. Unlike generalist code generation tools, CodiumAI reviews and understands code structure and desired functionality, considering metadata such as code comments while intelligently interacting with the developer via prompts in the IDE. 


  • Existing software testing solutions are outdated and ineffective. Code Integrity is a new paradigm that reimagines unit testing, code analysis, and code coverage: While basic testing frameworks exist, developers are responsible for manually implementing their test logic. Our conversations with them and their team leads validate the difficulty of writing effective and accurate unit tests using existing solutions. Moreover, developers spend >20% of their time writing tests, which is time not spent pushing new software. CodiumAI empowers developers to generate meaningful interactive test suites within their IDE by analyzing source code, docstring, and comments, equipping developers with code correctness and quality-checking capabilities at a level previously unavailable in an automated (i.e., generative) way. 

Commentary: Today, we define Code Integrity as (a) the code correctness-checking processes accompanied by (b) the metrics to measure the completeness of these code correctness-checking processes. While automated unit tests are a powerful method and wedge into (a), CodiumAI’s approach expands across static/dynamic code analysis, code review automation, and even comparing code behavior to written product specifications – further distinguishing it from existing testing solutions. We have also observed increasing demand from CTOs and team leads for improved code coverage measures across the organization through a centrally governed platform. Since our investment, it has become clear that developers and executives are demanding the tooling, automation, and unification of both (a) and (b). Expanding their enterprise offering, CodiumAI’s platform may include collaboration tools, test data management, CI/CD integration, auto-fixing of bugs, code improvement suggestions, and eventually, the enablement of next-generation, test-driven development.


  • Breakthrough generative technology paired with a world-class team: Until recently, LSTMs have been the state-of-the-art model architecture for code generation. The advent of the Transformer architecture unlocks tremendous scale and generalizability of Large Language Models (LLMs), delivering the most powerful code generation yet. Examples of recently trained code generation LLMs include (1) Deepmind’s AlphaCode, (2) OpenAI’s Codex, and (3) Facebook’s InCoder. While this technology will be impactful, leveraging it effectively is challenging and requires a combination of deep academic and commercial experience. Harnessing both of these, CodiumAI’s founding team are serial entrepreneurs who previously founded and exited startups and led product and R&D teams at leading organizations like Alibaba Cloud. The team started CodiumAI to address the chronic pain they dealt with throughout their careers: verifying and validating code.

Commentary: Improvements in the quality and accessibility of LLMs have been remarkable. Model layer providers such as OpenAI, Stability, Cohere, Adept, and AI21 have innovated across text, vision, and audio. While CodiumAI is incorporating existing state-of-the-art LLMs, the team is executing rapidly on training their own LLM and system, TestGPT. TestGPT specializes in code testing (including designated integrations and data digestion capabilities) and is trained on a large, proprietary test-related dataset. Aside from the LLM itself, CodiumAI’s team has built a significant engineering stack and designed a product experience (more below) that significantly supplements their model.

Primary Concerns

Below, we review the primary concerns we evaluated for our initial investment in CodiumAI, and discuss which issues are most relevant today:

  • Existing generative models may be incapable of generating sufficiently accurate unit tests.

Commentary: Though CodiumAI has benefitted from breakthroughs in code generation LLMs, it has become clear that generating accurate unit tests requires more than calling a “Language-Model-API.” Understanding this, CodiumAI’s team has developed a proprietary stack that builds around their TestGPT model. Elements of this stack include automating testing-domain prompting, parallelizing and chaining multiple prompts, and efficiently gathering broad code context for the prompts. Even with the evolution in LLMs, challenges remain in model grounding & referencing,  limited LLM context/input sizes, fine-tuning, and alignment. While these challenges may hold back the majority of engineers, we believe CodiumAI’s research-heavy team will make timely advancements that reinforce defensibility.


  • Well-funded generalist AI research companies such as OpenAI may enter the category and leverage capital as a moat to out-compete in talent and product development.

Commentary: We believe in the short to medium term, AI research companies will focus on model architecture R&D rather than building domain-specific commercial products. In this time, CodiumAI will set itself apart with a differentiated engineering stack and product experience. Already, the team’s focus on product design and implementation tailored to the developer has unlocked novel workflow and data moats. The synergy between (a) CodiumAI’s fine-tuned model (which ingests code, metadata, and documentation), (b) domain-specific engineering (context collection, prompt compression, chaining and parallelizing model calls), and (c) an interactive development-flow integrated with advanced analyses and metrics, will ultimately impact a developer’s confidence in a way that one-fit all code generation tools cannot achieve. ​​With thousands of developers already using the closed-alpha (made publicly available today, March 22, 2023), we believe CodiumAI’s continued success will stem from a combination of engineering, model, and product differentiation.


Note: This excerpt or attachment is a representative excerpt from Vine’s internal IC memo produced in June 2022. It should be used for information purposes only, and is not intended as an offer or commitment, a solicitation of an offer, investment advice, or recommendation to enter into any transaction.