concept datum in category edge computing

This is an excerpt from Manning's book Making Sense of Edge Computing.
Edge computing is the practice of moving data processing closer in proximity to the sources of data generation. The purpose of edge computing is to establish a decentralized and potentially hierarchical computational paradigm to support the development and operations of distributed systems. This practice can break down a centralized computing problem — usually one where you move all of your data to a central server for processing — into a much more efficient, distributed solution by doing some of the processing on devices connected to your network. This means instead of needing a large investment in the form of a data center or cloud infrastructure you can use those same devices you use to gather the data to perform some or all of your processing for you, saving you from having to send all of your data back to a central server or data center and reducing overall costs to boot!
Maybe you’re happy with your current setup, gathering all of your data to a local datacenter or to the “cloud,” where it can be easily dealt with. For smaller, less complex workflows this will work just fine. Consider, though, if you need to process something fairly large and continuous, like streams of video for unauthorized personnel. Can your centralized system handle 10 video streams, or hundreds, or thousands? What if those streams are in higher resolutions like 4K? With data feeds that large, you may find yourself struggling — or, worse, paying handsomely — to even transfer it all back to your data lake; let alone storing and processing it. With edge computing you handle the processing of these streams either on the devices themselves — or on infrastructure that is close in proximity to the devices with more adequate resources — meaning scalability is much easier to achieve and the hardware requirements are much easier to meet on a per-device basis. So rather than processing all those video streams in your data center, you tell each device to look for unauthorized personnel and only send alerts when they find one. Now you only need to transfer video data when you are reviewing and confirming an alert you’ve received, thus saving time and money on data transfers and infrastructure.
Maybe the volume isn’t so much of a concern, but rather that content of the data itself. What if your project contains private information — such as patient health data — that you may not be able to transfer off the source device? By processing the private data on the device, or on an adjacent device in the same location, and only transferring the results, you alleviate the need to transmit the original private or sensitive information unless absolutely required, ensuring security while still allowing utility out of potentially sensitive data.
In this chapter you’ll learn to identify streaming sources of data, recognize temporal events, and understand when and how to apply complex event-processing techniques to specific problems. The chapter is structured in a way so that you need no previous experience in streaming data or event processing. Streaming data concepts are covered, which leads in to streaming (event) processing. Figure 4.1 shows an event being extracted from a data stream. Building on event-processing concepts you’ll cover complex event processing involving, as the name suggests, several related event processors. Finally, you’ll learn by example how streaming complex event processing can be used within edge computing.
Figure 4.1 Raw data streams are often infinite and continuous flows of information. Events must be extracted to create event stream, which can then be manipulated my event processors.
![]()
From an edge-computing perspective, data streams and event processors can be managed as edge applications, where event processors are nodes and data paths are edges in an edge-computing network. Data generated on a low-powered device might makes its way through many nodes, being transformed or enriched along the way. As the number of event sources and data rates increase, you’ll have no choice but to apply some form of edge computing to projects where it is needed. That is to say, functionality must be move toward sources of data generation with appropriate data operations taking place along the data stream. These streams often form complex graphs, which are discussed in detail in chapter 5. These application graphs might have various complex event processors distributed across a large geographically distributed application, or a simple event processor used to alert on soil moisture events running on a single Raspberry Pi on behalf of your prize rose bushes. The first step in this journey is to better understand just what streaming data is anyway.
4.1.1 Data in transit versus data at rest
There are knowns, unknowns, and unknown unknowns. When dealing in hierarchical systems, such as the Open Systems Interconnection (OSI) network model, knowledge on a higher level might not make it to a lower one, and vice versa. For instance, the HTTP protocol used most web traffic provides transaction headers specifying exactly how much data is going to be transmitted and the format of the data. But lower-level network sockets moving the data around are provided no such information. After all, sockets provide the underlying data to inform the higher-level HTTP protocol, so who would tell them? Sockets can just deal with the unknown, it is above their pay grade, they don’t ask questions and do their job according to protocol. To better understand how streaming works, consider what programmatically takes place in network socket programming. Typically, socket programs are made up of servers and clients. On the server application a socket listener is instantiated on a specific port number, which starts listening for incoming data. On the client side, a connection is established with the host and port on the server (listening) side allowing for two-way communications. When a message is sent from the client to the server there is no need close the connection between host and in theory could be used to provide an infinite stream of data. Take a look at the following Java code example showing a server-side network socket listener implementation.
ServerSocket serverSocket = new ServerSocket(portNumber); #A Socket clientSocket = serverSocket.accept(); #B PrintWriter out = #C new PrintWriter(clientSocket.getOutputStream(), true); BufferedReader in = new BufferedReader( #D new InputStreamReader(clientSocket.getInputStream()));Network sockets are implemented using language-specific streaming functions that control the flow of data. Think for a moment about why streaming functions are used in network socket programming. When a client connects to a listening server socket, that is like your grandmother calling you on the phone. You don’t know if the call is really for you, how any words will be spoken, or if your grandmother will ever hang up the phone. Now consider receiving a birthday card from your grandmother. The birthday card is an immutable data structure preserved in time, addressed to you, and if you are lucky containing a crisp $5 bill. You know what to do with the birthday card, but the phone call is full of unknowns and like a network socket you must deal with it according to protocol and not forget to thank her for the $5.
Data is finite and continuous data or streams is infinite In order to make use of data in stream a finite sample data sample must be identified, and extraction methods developed.