Most engineers think…
Most people assume 'connecting a source to Sentinel' means dropping an agent on a machine and logs magically appear. That works for Windows events — but it completely misses the architecture for 400+ other source types.
Sentinel has three distinct ingestion paths: native first-party connectors (for Entra ID, Defender, Office 365), AMA-based CEF and Syslog connectors for Linux appliances and firewalls, and the Logs Ingestion REST API (with the Codeless Connector Platform for partners). Once data arrives, ASIM normalization maps it into standard schemas so one detection fires on any vendor's firewall. And once you have volume, you must choose between Analytics, Basic and Auxiliary table types to keep costs predictable — three table types and commitment tiers that most engineers discover only after a surprise bill.
① The three ingestion paths — native, AMA and REST
Microsoft Sentinel organises its 400+ connectors into a few clear families. Native (first-party) connectors are one-click for Microsoft's own services: Entra ID sign-in and audit logs, Microsoft Defender XDR incidents, Office 365 audit events, Azure Activity, and Defender for Cloud alerts. These use the Microsoft Graph or Azure Resource API under the hood — no agent needed.
For everything else, the second family is agent-based connectors. The Azure Monitor Agent (AMA) is the current standard. It collects Windows Security Events natively and powers the CEF and Syslog connectors for Linux appliances. The legacy Microsoft Monitoring Agent (MMA) is retired — any existing MMA deployments should migrate.
The third family is the Logs Ingestion REST API (formerly the HTTP Data Collector API, which retires after September 2026). Custom integrations, scripted pipelines, and third-party SaaS products that do not have an AMA-based connector use this path to POST JSON logs directly into a Log Analytics workspace table. The Codeless Connector Platform (CCP) wraps this API so partners can publish a connector to the Sentinel gallery without writing agent code.
Which connector family is the right choice for connecting Microsoft Entra ID sign-in logs?
② AMA, CEF & Syslog — the appliance log-collection path
Network appliances — firewalls, intrusion detection systems, proxies, VPN concentrators — speak either raw Syslog or CEF (Common Event Format). The AMA-based collection path works in two hops. First, the appliance sends its Syslog or CEF stream to a Linux forwarder VM running the local Syslog daemon (rsyslog or syslog-ng). Second, the Azure Monitor Agent on that forwarder reads from the daemon and pushes the data to the Sentinel workspace over HTTPS — no inbound port to Sentinel is required.
For raw Syslog, logs land in the Syslog table. For CEF, they land in CommonSecurityLog. The AMA replaces both the older OMS/MMA agent and the legacy Syslog connector; the Data Collection Rule (DCR) on the forwarder defines which facilities and severity levels to collect, so you are not paying to ingest debug-level noise unless you choose to.
Why the forwarder matters
The Linux forwarder VM is a single chokepoint — if it goes down, your firewall logs stop. Production deployments run at least two forwarders behind a load balancer. Size the forwarder to the peak events-per-second of your appliance fleet; the AMA is lightweight but the Syslog daemon can become the bottleneck at very high event rates.
The current recommended collector for Windows and Linux. Replaces the legacy MMA/OMS agent. Powered by Data Collection Rules (DCRs) that define what to collect and where to send it.
Common Event Format — a structured Syslog extension with key=value pairs. CEF logs from network appliances land in the CommonSecurityLog table in Log Analytics.
A JSON connector definition that tells Sentinel how to poll a REST API, authenticate, and write to a Log Analytics table — no agent code needed. Lets SaaS vendors self-publish to the Sentinel gallery.
A KQL function that maps raw vendor-specific table fields to a standard ASIM schema field name (e.g. SrcIpAddr). Write one detection rule against the ASIM schema and it works for every mapped source.
The Linux CEF/Syslog forwarder is a single point of failure. In production, deploy at least two forwarder VMs behind a load balancer and point your appliances at the VIP — if one forwarder goes down your firewall logs keep flowing. Size each forwarder to handle the full peak EPS of your appliance fleet, not just half.
▶ Watch a firewall CEF event flow into a Sentinel incident
End-to-end: Palo Alto firewall deny event arrives, traverses the forwarder, lands in CommonSecurityLog, hits an ASIM detection rule, and becomes an incident. Press Play for the healthy path, then Break it.
Where do CEF-format logs from a Palo Alto firewall land after AMA-based collection?
③ Codeless Connector Platform & ASIM normalization
The Codeless Connector Platform (CCP) lets anyone publish a Sentinel connector that pulls data from a REST API without writing a custom agent. You author a connector definition in JSON (specifying the API endpoint, authentication, polling interval, and the target Log Analytics table), publish it, and Sentinel's backend handles the scheduling and ingestion. The Sentinel connector catalog crossed 400 connectors partly because of CCP — SaaS vendors can self-serve their way into the gallery.
ASIM (Advanced Security Information Model) is the normalization layer that makes those 400 sources actually useful together. Every vendor calls the 'source IP' field something different. ASIM defines a set of standard schemas — Network Sessions, DNS, Web, Authentication, File Events, and more — and each schema has fixed field names. Parser functions map raw table fields (e.g. CommonSecurityLog, a Palo Alto table, a Cisco ASA table) to the ASIM schema. A Sentinel analytics rule written against _Im_NetworkSession fires on any vendor's firewall without vendor-specific edits.
ASIM parsers come in two flavours: built-in (native) parsers that run at query time and transform data on the fly, and ingestion-time transformations in a DCR that normalise before data lands in the table. Ingestion-time is faster to query but less flexible.
Avoid writing detection rules directly against raw vendor tables like CommonSecurityLog filtered by DeviceVendor='Palo Alto'. When you add a second firewall vendor you must duplicate the rule. Write against the ASIM _Im_NetworkSession or _Im_WebSession parsers instead — new vendor, new parser, zero rule changes.
A detection engineer wants a single Sentinel analytics rule to fire on network session data from Palo Alto, Fortinet, and Cisco simultaneously. What should she write the rule against?
④ Table types & commitment tiers — controlling what it costs
Log volume costs money, and Sentinel gives you three table types to manage that. Analytics tables are the default: full KQL query support, fast, indexed — and the most expensive. Use them for high-value security data you query every day (security events, CEF/firewall logs, identity logs). Basic tables are lower-cost with 8 days of interactive query and 90 days retention; they suit verbose-but-rarely-queried logs like verbose web proxy or application traces. Auxiliary tables go further still — cheapest ingestion, long retention, query via Search jobs only — for bulk data you keep for compliance but almost never touch interactively.
On top of table types, Sentinel charges ingestion per GB via two billing models. Pay-as-you-go is flexible but expensive at volume. Commitment tiers start at 100 GB/day and give a substantial per-GB discount in exchange for a minimum daily commitment — predictable cost for workspaces with stable ingestion. You can increase your tier at any time; lowering it is restricted to once every 31 days.
Architect for cost from day one
Route verbose, low-priority log types to Basic or Auxiliary from the start — it is far cheaper than ingesting everything into Analytics and then realising the bill. Use DCR filtering (on the AMA or the Logs Ingestion API) to drop log levels (e.g. Syslog debug and info) before they even reach Sentinel. Commit to a tier once your baseline stabilises, and revisit quarterly.
Priya at a Mumbai fintech firm faces this
After enabling 15 new connectors for network firewalls and web proxies, the Sentinel monthly bill doubles overnight with no corresponding increase in detected threats.
All logs — including verbose Syslog debug messages and web proxy access logs queried only during quarterly audits — were ingested into Analytics (full-cost) tables using default connector settings.
Cost management blade in Sentinel: Analytics table ingestion shows the proxy and debug Syslog streams are the top cost contributors but appear in zero detection rules.
Sentinel ▸ Settings ▸ Workspace Settings ▸ Tables + DCR configuration for each connectorMove verbose proxy logs to Basic tables. Add a DCR filter on the Syslog AMA connector to drop debug and info severity levels. Evaluate a 200 GB/day commitment tier for the remaining high-value streams.
Week-on-week cost in Azure Cost Management drops by roughly 40%; critical security alerts (firewall deny events, CEF malware detections) continue to fire normally from Analytics tables.
After adding a Syslog severity filter in a DCR, run a quick KQL on the Syslog table: Syslog | where SeverityLevel == 'debug' | count. If you still see debug rows, the DCR change has not propagated or is targeting the wrong machine group — check DCR association in Azure Monitor before assuming cost savings.
Your workspace ingests 500 GB/day. Azure Monitor verbose proxy logs are queried maybe once a quarter. What is the best cost approach?
🤖 Ask the AI Tutor
Tap any question — instant, scoped to this lesson. No login, no waiting.
Pre-curated from vendor docs + community Q&A, scoped to this lesson. For a live prod issue, paste your export into chat.techclick.in.
📝 Wrap-up assessment — six more
You've answered 4 inline. Six left. 70% (7 of 10) marks the lesson complete on your profile. Tap Submit all answers at the end.
🧠 In your own words
Type one line: why does an ASIM parser mean you never need to rewrite a detection rule when you add a new firewall vendor? Then compare with the expert version.
🗣 Teach a friend
Best way to lock it in — explain it in one line to a teammate. Tap to generate a paste-ready summary.
📖 Glossary
- Azure Monitor Agent (AMA)
- The current recommended log collection agent for Windows and Linux. Replaces the legacy MMA/OMS agent. Configured via Data Collection Rules (DCRs) that filter, transform and route data before ingestion.
- CEF (Common Event Format)
- An industry-standard Syslog extension with structured key=value pairs, supported by most security appliances. CEF logs collected by AMA land in the CommonSecurityLog table in Log Analytics.
- Data Collection Rule (DCR)
- An Azure resource that defines what data the AMA collects (event channels, Syslog facilities, severity levels), any transformations to apply, and which Log Analytics workspace to send it to.
- Codeless Connector Platform (CCP)
- A JSON-based framework that lets partners and customers publish Sentinel connectors for REST-API data sources without writing agent code. Sentinel's backend handles polling and ingestion.
- ASIM (Advanced Security Info Model)
- Sentinel's normalization layer. Parser functions map raw vendor-specific table fields to standard schema field names (e.g. SrcIpAddr, DstPort) so one detection rule covers all ASIM-mapped sources.
- Analytics table
- The default Log Analytics table type: full KQL, indexed, highest query performance, and highest ingestion cost. Used for security data queried in daily detections and investigations.
- Basic / Auxiliary table
- Lower-cost table types. Basic supports 8-day interactive KQL and 90-day retention. Auxiliary is cheapest with long retention but requires Search jobs for ad-hoc queries — used for verbose compliance logs.
- Commitment tier
- A Sentinel/Log Analytics billing option that commits to a minimum daily ingestion (starting at 100 GB/day) in exchange for a lower per-GB rate than pay-as-you-go pricing.
- Linux CEF forwarder
- A Linux VM running rsyslog or syslog-ng plus the AMA, which receives CEF/Syslog from network appliances on UDP/TCP 514 and forwards events to the Log Analytics workspace over HTTPS.
📚 Sources
- Microsoft Learn — Microsoft Sentinel data connectors overview. learn.microsoft.com/en-us/azure/sentinel/connect-data-sources
- Microsoft Learn — Syslog and CEF AMA connectors overview. learn.microsoft.com/en-us/azure/sentinel/cef-syslog-ama-overview
- Microsoft Learn — Log retention tiers in Microsoft Sentinel (Analytics, Basic, Auxiliary). learn.microsoft.com/en-us/azure/sentinel/log-plans
- Microsoft Learn — Plan costs and understand pricing and billing — Microsoft Sentinel. learn.microsoft.com/en-us/azure/sentinel/billing
- Microsoft Tech Community — What's new in Microsoft Sentinel: May 2026. techcommunity.microsoft.com/blog/microsoftsentinelblog
- Microsoft Learn — ASIM (Advanced Security Information Model) overview. learn.microsoft.com/en-us/azure/sentinel/normalization
What's next?
Got the ingestion pipeline locked down? Next, go deep on Sentinel Analytics rules — scheduled queries, NRT rules, Fusion ML detections, and how to tune a rule without drowning the SOC in low-fidelity alerts.