6 things you need to know about Microsoft streaminsight

Link: http://seroter.wordpress.com/2010/06/22/6-things-to-know-about-microsoft-streaminsight/

Microsoft streaminsight is a new product in SQL Server 2008 R2. It is Microsoft's first involvementEvent stream processingAndComplex Event ProcessingThe product launched by the market, although it already has a share of mature products and thought leaders. I spent more than 8 months on this product. I think I should try to tell you something you need to know about streaminsight.

• Event Processing focuses on continuous intelligence.
  EventIt can be a variety of things: it can be a change in the user's address, or it can be a read degree from the electric meter. When you have an event-driven architecture, you will deal with asynchronous data communication, because it occurs when the consumer chooses what action to take based on the data. The so-called "complex event processing" refers to collecting knowledge from multiple (simple) business events and summarizing it into a smaller collection of events. Streaminsight can be used to connect data from multiple data streams and detect event modes that are visible through group intelligence. Unlike traditional database-driven applications that submit queries continuously in the standard data set, the event processing solution deploys a series of compiled queries when the event data goes through. This is a pattern change for most people, and it is not easy to change it at half past one, but there is no doubt that it has received much attention for improving the enterprise's business intelligence strategy and improving information availability.
• Write a query using LINQ.
  The streaminsight group selects LINQ as the mechanism for compiling declarative queries. As expected, you can write different queries to filter content, connect different data streams, and execute computing. If I want the customer's call center to quickly send an event when a specific product name appears in each customer complaint, what should I do? The following query can filter out all the other products mentioned and focus on the events of the target product:

  var filterQuery =    from e in callCenterInputStream    where e.Product == "Seroterum" select e;
  

  An important aspect of streaminsight query is aggregation ). The calculation and filtering of a single event are acceptable, but suppose what we want to know is the result of a period of time? This timeWindow(Windows) concept to play a role. If you want to perform one count, average value, or event aggregation, you only need to specify a window of interest for a specific period of time. For example, if you want to know the most popular page of a website in the past 15 minutes and want to recalculate it every other minute, that is, the number of clicks per page in the past 15 minutes is calculated every minute. This is the hopping window ).

  var activeSessions = from w in websiteInputStream                     group w by w.PageName into pageGroup                     from x in pageGroup.HoppingWindow(                         TimeSpan.FromMinutes(15),                         TimeSpan.FromMinutes(1),                         HoppingWindowOutputPolicy.ClipToWindowEnd)                     select new PageSummarySummary                     {                         PageName = pageGroup.Key,                         TotalRequests = x.Count()                     };

  I will introduce more about this topic in the next blog, but now you only need to know that there are other types of windows in streaminsight.

  I strongly recommend that you read the streaminsight group's articles on this topic.

• The query can be used repeatedly and can be linked to each other.
  A good solution for event processing is that it can be linked for query. Consider this scenario: the first query accepts thousands of events per second and gets only a subset of events I Care About When filtering noise. Then, I can use the output of this query in another query for computing or aggregation in this more targeted event stream. You can also consider the scenario of "pub/Sub" subscription to obtain data flow events from a data source, but there are multiple output targets. A query can obtain results from a stream and apply the results to many other streams.

• Streaminsight uses an adapter model for input and output data.
  The result of building a streaminsight solution is often changed to creating or using an adapter. Although streaminsight itself does not come with any product-level adapters, we are fortunate to find many examples of best practices. In my upcoming new book, I will show you how to build an MSMQ adapter: read data from the queue and send it to the streaminsight engine. The adapter can be compiled in a generic or non-typed manner for simple reuse, or by adding the expected specific event load. As expected, writing specific adapters is simpler, but building reusable adapters is obviously more beneficial in the long term.

• Select multiple hosts.
  If you want to select a host, you can choose to create a streaminsight server in the same process to host the query and use the adapter to connect to the data publisher and user. This is almost the easiest way to build streaminsight and the most controllable engine solution. Another option is to install streaminsight as a Windows service on your machine and use it as the central server. The former uses "server. Create ()" for operations, and the latter uses "server. Connect ()" for interaction with the engine. I am writing a blog on how to use remote server options. Please stay tuned. Now you only need to know how to select the host.

• StreaminsightDebugging tools are good, but the overall management function is not mature enough.
  Streaminsight has released a very interesting debugging tool, which is also the only graphical interface that can be used to initially manage servers ., When you connect to a server (whether an independent process or a host type), you can see the deployed applications and queries.
  
  When a query is running, you can choose to record the activity and replay the stream. When you see that your queries are processed through various LINQ operations (such as connections and counts), it must be nice!
  
  The debugger contains many good problem root cause analysis capabilities and tracking events in the query step. You can also obtain many server-side diagnostic information about engines and queries. However, no other graphical tools can be used to manage servers. Of course, you will find that you can execute other management tasks by writing code or using powershell. I very much look forward to filling this gap with examples provided by Community tools and product groups, as well as a more robust management interface for products to be released in the future.

The above is a brief introduction to streaminsight. If you want to learn more, you can read the section about streaminsight in my new book, in addition, I maintain a page about streaminsight resources on the book's website.