Is Seq an alternative to Application Insights?

Seq and Application Insights both collect application logs in Azure. Despite this overlap, Application Insights and Seq have different objectives:

Application Insights is a data platform that addresses a wide range of operational monitoring scenarios
Seq is focused on the needs of software development teams monitoring and debugging their own apps and services

The rest of this article will compare Seq with Log Analytics, the component of Application Insights and Azure Monitor that provides log search and analysis, and demonstrate why Seq is often preferred when diagnosing application issues.

Querying and data model compared

How a system captures and represents event data, and how that data can be manipulated, makes a big difference to how easy it is to find what you are looking for.

Both products include log filtering and aggregation capabilities. Seq has the Seq Query Language, while Log Analytics has the "Kusto" query language.

Seq supports structured logging, via the message template standard. This allows Seq to capture events as fully-structured objects with properties that can be compared and used in calculations. Event properties are accessed directly with expressions like:

Product.Name

Log Analytics has a fixed schema so it must store event properties as a JSON string in a column called customDimensions. Before using event data you must first cast and parse the value, which is why accessing a property of an event tends to require expressions like:

todynamic(tostring(customDimensions.Product)).Name

These ergonomic differences between Seq and Log Analytics reflect Seq's focus on the application diagnostic experience, and Application Insights' aim to be a single, all-encompassing operational monitoring tool.

The example data set used in this comparison is generated using the Serilog sinks for Seq and Application Insights. The screenshots below show the same events recorded in each system; in Log Analytics:

Viewing an event in App Insights Log Analytics

And in Seq:

Viewing an event in Seq

To understand the implications of the different approaches we will examine the ergonomics of querying structured log data with each product.

Scenario #1: Find all events for the product 'Rocket Ship Dark Roast, Ground'

One of the simplest requirements when analyzing application logs is to filter the log by an event property. In this case, we wish to filter the log down to all events with a product named 'Rocket Ship Dark Roast, Ground'.

Here is the Kusto query for Log Analytics:

traces
  | where todynamic(tostring(customDimensions.Product)).Name
  == "Rocket Ship Dark Roast, Ground"

Because Log Analytics stores all custom properties as strings, the customDimensions.Product value has to be cast to a string and then to dynamic, before the Name property can be extracted. The awkward syntax is due to Log Analytics needing to accommodate structured logging in a data model originally designed for unstructured or semi-structured events.

The Seq query language equivalent is:

Product.Name = 'Rocket Ship Dark Roast, Ground'

This works because Seq is built specifically for structured log data. The substantial character count difference here becomes apparent during interactive debugging, where the Seq syntax is considerably more efficient.

Scenario #2: Sum of the weight of each product per hour

This query computes the weight of coffee sold, for each product, for each hour. Using Log Analytics:

traces
  | extend Product = todynamic(tostring(customDimensions.Product))
  | where isnotnull(Product.Name)
  | summarize sum(toint(Product.SizeInGrams)) by strcat(
      format_datetime(timestamp, 'yyyy-MM-dd'), 
      " ", 
      datetime_part("Hour", timestamp)), 
      tostring(Product.Name)
  | order by Column1

Product is declared to reduce duplication. The keyword summarize is the Kusto equivalent of group by. Because Kusto doesn't have a built-in way to group by time intervals, it is necessary to construct a string that identifies the hour of the day in which the event occurred.

The Seq Query Language expresses this query with SQL-style syntax, leveraging the time() built-in function to group results by time interval:

select sum(Product.SizeInGrams) 
from stream 
where Product.Name is not null 
group by time(1h), Product.Name

For software developers investigating the behavior of a system over time, queries like this are common, so Seq works hard to ensure the query can be expressed clearly and with minimal effort.

Querying and data model, key take-aways

Seq's data model fully embraces structured log events. This greatly improves the ergonomics of constructing ad-hoc queries, which is important for real-time investigation and diagnostics. Seq builds on this data model with a terse, familiar, SQL-style syntax that is optimized for efficient interactive use.

Application Insights uses a more general, flexible data model, at the cost of additional complexity when searching and analyzing logs. Application Insight's Kusto query language has powerful analytic features, but these too add complexity and cost to the user.

The two small examples discussed here illustrate the different goals and perspectives that distinguish Seq from Application Insights. While Application Insights has great facilities for general data manipulation, Seq shines when ease-of-use and efficiency of expression are paramount.

Data retention compared

Data retention determines how long log events are kept available for searches and queries. This often presents a conflict between space and cost, on the one hand, and potential value for diagnostics and analysis on the other.

Log Analytics retains event data for 31 days. If you choose to retain data longer there is a cost ($0.15/GB/month at the time of writing^). There is no fine-grained control of retention to be able to specify different lifespans for different log data. Coarse-grained control of retention data is only available via the API.

Seq retention is determined by 'retention policies' which allow fine-grained control of data retention. High-volume events can often be removed after a short period of time, while other business-relevant events might be retained longer. There is no restriction on how long you can keep log events, except the resources of the underlying infrastructure.

Seq Storage screen showing retention policies that remove targeted events at configured intervals

Data retention key take-aways

Seq's fine-grained retention policies provide more flexible data retention than Log Analytics does. Coupled with no per-GB charges, this helps teams strike a better balance between the value of retained log data, and the cost of storage.

Hosting options

Log Analytics is a cloud service only. It can be deployed to your choice of Azure availability zone.

Seq is a self-hosted software product. It must be installed on a server, which might be within your local network, within a data center or within a cloud. Users have complete control of their data and the ability to comply with a wide range of data privacy and security requirements.

Included support

Log Analytics requires the additional purchase of an Azure support plan ($1,200/year^), on top of the cost of Application Insights itself. This provides access to the Azure support team.

Seq support is included in every Seq subscription, and is provided by the engineers that build the product.

Cost

There is no exact comparison of costs because it depends so much on configuration and specific circumstances. However, it is possible to make some useful generalizations to arrive at a ballbark estimate.

Log Analytics is priced primarily on data ingestion, currently $3.34/GB^, with 30 day retention. Log data retained beyond thirty days costs $1.80/GB/year^. Log data exported to another location within Azure costs $0.145/GB^.

Seq is priced at $690 - $7,990/year^ depending upon the number of users and installations required. In addition to the subscription cost, hosting infrastructure needs to be priced in. The hardware required will depend upon the volume of data to be ingested, searched, and retained.

The following cost comparison shows how the cost of Seq and Log Analytics typically varies with the amount of data ingested. This comparison assumes 30 day log retention and includes subscription costs, hardware costs (for Seq) and support costs. The hardware costs for Seq are based on reasonable on-demand virtual machine pricing from Azure and AWS.

Chart showing that Log Analytics costs grow much faster than Seq costs

Extensibility

Log Analytics, and Application Insights, include a lot of functionality. There are no extensibility points.

Seq has a thriving ecosystem of "Seq Apps" that are used to extend the functionality of Seq.

List of Seq apps on nuget

Seq Apps do things like connecting to Slack and Email, ingesting data from different sources, and archiving logs to files. It is also possible to implement your own Seq Apps.

Conclusion

Seq is designed and built to be the ideal tool for software development diagnostics. Application Insights is a broad operational monitoring platform. Seq is a better tool if your goal is to understand the behavior of complex application logic and system interactions.

Microsoft has made Application Insights the default choice for logging in Azure by automatically provisioning it, however, the small extra effort to use Seq instead provides a better diagnostic experience and in many cases a smaller bill.

^ All pricing mentioned here is approximate. It represents the best information available at the time, but is subject to change, regional differences and variations based on circumstance.