Skip to content
DrCompsSignal / computer news
Menu

Confluent turns Kafka topics into sub-second SQL tables

Lightning Tables materialize Kafka topics for SQL and MCP queries, but the first release is AWS-only and schema-gated.

Confluent Cloud has added Lightning Tables, a managed layer that turns an Apache Kafka topic into a table for low-latency SQL queries. The architectural change matters because applications, analysts, and AI agents can query current stream data without first building and operating a separate consumer, Flink job, or serving database.

What changed

When Lightning Tables is enabled for a topic, Confluent materializes the records into a query-optimized table and says new topic data is reflected within 100 milliseconds. Queries can run through the Confluent Cloud console, an HTTP API, the Confluent CLI, JDBC, or a Model Context Protocol client.

The service supports two data models. Append mode retains every record for event-history queries. Upsert mode follows compacted-topic semantics, keeping the latest value for each key and treating a tombstone as a delete. Confluent selects the mode from the topic’s cleanup.policy.

The practical consequence

Teams can expose operational Kafka state to SQL tools and agents with less infrastructure and fewer data copies. That can shorten the path from an event stream to dashboards, support tools, and retrieval workflows. Access is governed through Confluent role-based access control, while compute and storage are billed by usage.

Limits and evidence

Lightning Tables currently require Confluent Cloud on AWS and a Basic, Standard, Enterprise, or Dedicated cluster in a supported region. Topics must have a registered Avro, Protobuf, or JSON Schema. Upsert mode additionally requires a compacted topic and a record key, and key-schema changes are not supported.

The console and CLI cannot query Lightning Tables over a private network; private deployments must use the API, MCP, or JDBC through a private endpoint. Confluent’s sub-second response and 100-millisecond update figures are vendor claims. DrComps has not independently reproduced the latency, throughput, availability, or cost behavior, which will depend on query shape, data volume, region, and usage.