AI libraries, from requirement to responsible choice
Imagine a team planning a support assistant. One engineer proposes LangChain. Another proposes CrewAI. A third wants LangGraph, Ragas, Promptfoo, Langfuse, and Phoenix.
The tool list grows before the team has written the problem.
Start with the jobs instead:
- call a model and retrieve policy documents;
- pause a claims workflow for human approval;
- compare a new prompt with the accepted version;
- trace a bad production answer.
These are different jobs. They may need different tools, or ordinary application code may already solve them.
Someone says, “We use CrewAI.” That tells us less than it first appears. It does not tell us whether the application has safe permissions, useful evaluations, recoverable state, or a reliable release process. It tells us which vocabulary and runtime the team chose for part of the system.
The name you heard as “Crevo AI” is most likely CrewAI. CrewAI organizes work around agents, tasks, crews, and flows. A crew is a group of role-based agents. A flow is the surrounding event-driven process that carries state and chooses routes.1
What you will understand
Section titled “What you will understand”| Chapter | Question you will be able to answer |
|---|---|
| 1. Categories | Which job is the library being hired to do? |
| 2. Frameworks | Which control model fits the application? |
| 3. Evaluation | Which failure must the test harness catch? |
| 4. Prompting | How should prompts be versioned and compared? |
| 5. Adoption | Which questions expose unnecessary lock-in or risk? |
Chapter 1: Put each library on the right shelf
Section titled “Chapter 1: Put each library on the right shelf”| Shelf | Question it answers | Examples |
|---|---|---|
| Application framework | How do model calls, tools, retrieval, and state fit together? | LangChain, LangGraph, LlamaIndex |
| Agent framework | How do one or more agents receive work and coordinate? | CrewAI, AutoGen, Semantic Kernel |
| Evaluation harness | Did this version pass the same cases as the last version? | Ragas, DeepEval, Promptfoo |
| Prompt and observability platform | Which prompt ran, what happened, and where did it fail? | LangSmith, Langfuse, Phoenix |
One product may sit on two shelves. That is normal. The categories describe the job you are buying it for, not the logo on its homepage.
Chapter 2: Choose a framework from the control model
Section titled “Chapter 2: Choose a framework from the control model”| Framework | Reach for it when… | Mental model | Official documentation |
|---|---|---|---|
| LangChain | You want common interfaces for models, messages, tools, retrievers, and runnable composition | Components and runnables | LangChain docs |
| LangGraph | The workflow branches, pauses, resumes, retries, or persists state | A state graph | LangGraph docs |
| CrewAI | Role-based agents and task delegation are central to the design | Agents + tasks + crews, surrounded by flows | CrewAI docs |
| LlamaIndex | Data ingestion, indexes, retrieval, query engines, and data-backed agents are central | Data framework with agent workflows | LlamaIndex agent docs |
| AutoGen | You are exploring conversational single-agent or multi-agent patterns and event-driven runtimes | AgentChat over an event-driven core | AutoGen docs |
| Semantic Kernel | A .NET, Python, or Java application needs model connectors, plugins, and agent orchestration | An application SDK with agent abstractions | Semantic Kernel agents |
| DSPy | You want to optimize a model program against examples and a metric instead of hand-editing prompt strings | Signatures, modules, optimizers | DSPy docs |
A real choice
Section titled “A real choice”A support-ticket classifier has one model call and a strict JSON schema. Plain provider SDK code may be enough. A claims workflow that retrieves policy, calls two services, waits for an adjuster, and resumes tomorrow benefits from explicit state and persistence. The second problem can justify LangGraph; the first may not justify any orchestration framework.
Chapter 3: Choose evaluation from the failure
Section titled “Chapter 3: Choose evaluation from the failure”| Tool | Best first use | What it does not replace | Official documentation |
|---|---|---|---|
| Ragas | RAG and agent metrics such as context recall, faithfulness, response relevance, and tool-call accuracy | Your labeled retrieval judgments and business rules | Ragas metrics |
| DeepEval | Python/pytest-style tests for outputs, RAG, agents, tools, conversations, and traced components | Calibrating model judges against humans | DeepEval docs |
| Promptfoo | Configuration-driven prompt/model comparisons, assertions, regression tests, and red-team cases | Production traces and application authorization tests | Promptfoo docs |
| LangSmith | Datasets, experiments, evaluators, human review, and traces around LangChain or other applications | Deterministic application policy | LangSmith evaluation |
| Phoenix | OpenTelemetry/OpenInference tracing, datasets, experiments, prompt work, and evaluators | A test dataset that represents your users | Phoenix docs |
| Langfuse | Traces, prompt versions, experiments, datasets, annotation, and online evaluation | The decision about what “good” means | Langfuse docs |
Do not begin with “Which evaluator has the most metrics?” Begin with one failure:
- Write the case. Include input, expected behavior, source evidence, and risk label.
- Use deterministic checks first. Schema, exact IDs, permissions, tool arguments, and citations often need no model judge.
- Add semantic scoring where rules are insufficient. Calibrate the judge on examples scored by people.
- Inspect disagreements. A score without the failing cases does not explain a regression.
- Gate the release. A cross-tenant leak should fail the build even when the average score is high.
Chapter 4: Treat prompting as a versioned program
Section titled “Chapter 4: Treat prompting as a versioned program”Prompt work has four separate jobs:
| Job | Practical options | Evidence to keep |
|---|---|---|
| Template and composition | Plain files, LangChain prompt templates, DSPy signatures | Template version and variables |
| Version and collaboration | LangSmith Prompt Hub, Langfuse prompt management, Phoenix prompts | Immutable version used by each run |
| Compare variants | Promptfoo, LangSmith experiments, Langfuse experiments, Phoenix datasets | Same cases, model settings, and evaluator |
| Observe production | LangSmith, Langfuse, Phoenix | Trace, tokens, latency, output, feedback |
The prompt is only one input to the application. Record the retrieved context, tool schemas, model and index versions too. Otherwise a prompt comparison may really be a data or retrieval comparison.
Chapter 5: Run a five-question adoption test
Section titled “Chapter 5: Run a five-question adoption test”Before adding any library, answer:
- Which concrete job will it own?
- Can the team export prompts, traces, and datasets?
- Where does sensitive text travel and remain?
- What happens when the library or its hosted service is unavailable?
- Which representative evaluation proves the integration helped?
If those answers are vague, keep the experiment behind a small adapter. Framework choice is reversible only when business logic and data contracts do not leak into every library-specific object.
Interview answer in 30 seconds
Section titled “Interview answer in 30 seconds”I choose AI libraries by the job they must own. LangChain standardizes model, message, tool, and retrieval interfaces. LangGraph makes stateful control flow explicit. CrewAI and AutoGen offer different agent coordination models. Ragas, DeepEval, and Promptfoo help run repeatable evaluations. LangSmith, Langfuse, and Phoenix overlap across traces, prompts, datasets, and evaluation. I start with the smallest tool, keep business rules behind my own interfaces, and verify that the new dependency improves a representative evaluation.
Continue with reranking models or classification metrics.
Footnotes
Section titled “Footnotes”-
CrewAI, official documentation, including its overview of agents, crews, tasks, and flows. ↩