Let's talk
WhatsApp +1 (201) 777-0433 LinkedIn /in/iskuzer Email iskuzer@gmail.com
Let's talk
Approach · Local & Private AI

Private AI

models that run inside your network

For companies that can't send client data to a public cloud. The same RAG, MCP and agents I run in WorkZeal, with the model moved onto your own servers and a gateway that decides what may leave.

See the setup Discuss your AI project
  • 0client records sent to public clouds
  • 1OpenAI-compatible API, so existing code keeps working
  • 2routes: sensitive local, general public
  • 100%of requests in your own audit log
Local & private AI · on-premise LLM

What stays inside your network

The model, the search index and the logs all run on your hardware. A gateway in front of them checks every request: anything with client or personal data is answered locally, and only what your policy allows may go to a cloud model.

YOUR NETWORK PUBLIC CLOUD Employees & apps chat · CRM · scripts AI gateway policy · data masking per-user keys · quotas audit log Local LLM · vLLM open-weight model · OpenAI API Your documents files · mail · wiki Private RAG local embeddings · pgvector MCP servers CRM · ERP · accounting · files Runs on your hardware a GPU server in your rack · a private cloud in your country · a Mac Studio for a small team Cloud model GPT · Claude · Gemini NEVER RECEIVES client records personal data contracts, prices your document index ask route index search sections tool calls live data only if policy allows · names and numbers masked
Everything in the green zone is yours: model weights, search index, prompts and answers. The gateway is the only door out, and it can stay closed. Apps talk to it like they talk to OpenAI, so switching an existing tool to the private setup is a one-line URL change.

Why run AI privately

Data never leavesclient records, contracts and personal data stay under your law and your contracts
No vendor lock-inopen-weight models you can swap, pin or roll back; no surprise model changes
Fixed costhardware instead of per-token bills, which pays off for heavy daily use

How I set it up

1Map the data

Which systems hold client or personal data, who may see what, what the law and your contracts say. This becomes the gateway's policy, not a slide.

2Pick models by task

Test open-weight models (Qwen, Llama, Mistral, gpt-oss) on your own questions and pick the smallest one that answers well. Small models for search and sorting, a larger one for writing and reasoning.

3Serve and connect

  • vLLM or Ollama with an OpenAI-compatible API
  • Local embeddings and pgvector for private RAG
  • MCP servers for CRM, ERP and files, with each user's own rights

4Gateway and logs

  • Per-user keys and quotas
  • Personal-data detection and masking
  • Local or cloud routing by policy
  • Every request in the audit log

Sizing the hardware

Small teamOne workstationA Mac Studio or one 24–48 GB GPU runs 8–32B models with quantization: chat, search and drafting for a team of 10–20.
CompanyGPU serverOne or two 80 GB GPUs run 70B-class models at full speed for a few hundred users, plus embeddings and RAG on the same box.
RegulatedPrivate cloud or clusterSeveral nodes with failover, separate environments and keys per department, monitoring and updates you control.

Rough starting points: the real numbers come from step 2, measured on your own questions and load.

Where it fits

Law firms and financecontract review and client mail without sending a single document out
Healthcarenotes and records stay in the clinic's network
Data-residency rulesdata must stay in the country, so the model moves there instead
Factories and offline sitesAI that keeps working with no internet at all