Adding AI to a Laravel app without making it a liability

Most AI features that disappoint do not fail because the model is bad. They fail because the feature around the model is vague. Nobody decided what a good answer looks like, what happens when the answer is wrong, or how anyone would find out.

These are the habits I use when adding a model to an existing Laravel application. They apply to any provider, and none of them depend on a particular SDK.

Pick a job small enough to check

"Chat with your data" is a tempting first feature and a hard one to get right. Better first jobs are narrow and checkable: sort incoming support tickets into eight categories, pull the invoice number and date out of an email, suggest a title for a draft. A person can look at the result and say yes or no in a second, which also means you can measure it.

Ask for structure, then do not trust it

Ask the model for JSON that matches a schema, and then validate it with the same tools you use for any other untrusted input. Laravel's validator is perfectly good at this.

final class TicketTriage
{
    public function __construct(private ModelClient $model) {}

    public function classify(Ticket $ticket): void
    {
        // ModelClient wraps whichever provider SDK you use
        $raw  = $this->model->complete('ticket-triage@v3', $ticket->body);
        $data = json_decode($raw, true);

        $validator = Validator::make(is_array($data) ? $data : [], [
            'category'   => ['required', Rule::in(TicketCategory::values())],
            'confidence' => ['required', 'numeric', 'between:0,1'],
            'reason'     => ['required', 'string', 'max:300'],
        ]);

        if ($validator->fails() || $data['confidence'] < 0.75) {
            $ticket->update(['triage_status' => 'needs_review']); // a person decides
            return;
        }

        $ticket->update([
            'category'      => $data['category'],
            'triage_status' => 'suggested', // a suggestion, never applied silently
        ]);
    }
}

Three things are happening here. The output is validated against fixed rules, so a made-up category is rejected. Anything uncertain goes to a person rather than being applied. And what the model produces is stored as a suggestion, which the interface presents as one.

A word on that confidence number: a model's own estimate of how sure it is has no guarantee of being well calibrated. I use it as a cheap gate to send doubtful items to review, and I judge the feature on how often people accept the suggestion, not on the number.

Keep it off the request path

Model calls are slow and sometimes fail. Put them in a queue so the page the user is on never waits for one, and design the feature so the product still works when the model is down.

// Off the request path: the ticket saves instantly, the suggestion arrives when it arrives.
final class TriageTicket implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable;

    public int $tries = 2;
    public int $timeout = 30;

    public function __construct(private readonly int $ticketId) {}

    public function handle(TicketTriage $triage): void
    {
        $triage->classify(Ticket::findOrFail($this->ticketId));
    }

    public function failed(Throwable $e): void
    {
        Ticket::whereKey($this->ticketId)->update(['triage_status' => 'needs_review']);
    }
}

If the job fails, the ticket falls back to the human process it already had. That fallback is the difference between a feature and a dependency.

Decide where a human stays in the loop

The question I ask is: how expensive is it when this is wrong, and how easily can it be undone? Re-tagging a ticket is cheap and reversible, so a suggestion with one click to accept is fine. Sending a message to a customer or changing a payment is neither, so those stay behind an explicit approval.

Log what you would need to debug it

For every model call I keep a row with the prompt version, the model, the (redacted) input, the raw output, latency, token counts, and, once a person has acted on it, whether they accepted, edited or rejected the suggestion. The accept and edit rates are the honest scoreboard. When someone says "the AI got that wrong last Tuesday", you can find out what actually happened.

Test with real examples, every time the prompt changes

Collect fifty to a hundred real inputs with the answers a person agreed were right. Run them whenever the prompt or the model changes, and compare. It does not need to be fancy: a script, a table of results and a diff. Without it, every prompt tweak is a coin flip you cannot see.

Be careful about what leaves your system

• Send the minimum. If the task does not need a customer's name or address, strip it first.
• Read the provider's data-handling terms for the plan you are on, and make sure they fit your own obligations.
• If you add semantic search or retrieval, apply the same permission checks as the rest of the app. A vector index that ignores who is allowed to see what will happily summarize another tenant's documents.

Put a ceiling on cost

Cache identical inputs, cap the length you send, and set a budget per tenant or per day with an alert before the limit. A loop that retries on error against a paid API is a bill waiting to happen.

And sometimes: do not

If a handful of rules or a SQL query solves the problem, use those. They are cheaper, faster, and fully explainable. A model earns its place when the input is messy, human language and the rules would be endless.