ARS

Surprise! Uber and Lyft don’t like NYC’s new ride-hail rules

Uber CEO Dara Khosrowshahi, who took the helm of the controversial company back in 2017, is known for being pretty unflappable. He was even upbeat during the company’s second quarter earnings call, when he was charged with explaining why Uber posted more than $5 billion in losses
/
Accenture News

Accenture Report Reveals New Cybercrime Operating Model Among High-Profile Threat Groups

Accenture Report Reveals New Cybercrime Operating Model Accenture Security’s 2019 Cyber Threatscape Report identifies top threats influencing the cyber landscape, including emerging disinformation techniques and evolving relationships in the underground economy ARLINGTON, Va.; Aug. 14, 2019 – Cybercrime campaigns and high-profile advanced persistent threat groups
/

Design Practices: AWS IoT Solutions

AWS IoT Solutions

Internet of Things (IoT) presents an unparalleled opportunity for every industry to address their business challenges. With the proliferation of devices, one needs a solution to connect, collect, store, and analyze the device data. Amazon Web Services provides various services that help connected devices to easily and securely interact with cloud applications and other devices for various user scenarios. Having said this, every Solution Architect in the field knows the capabilities and reliabilities of AWS Cloud. Migrating or designing Internet of Things (IoT) solutions on to AWS platform enables one to focus on core business without the hassle of infrastructure management and monitoring. This will ensure high availability to the customers. No matter whichever solution is designed, one should select the best platform to keep the solution stable. AWS is one such platform.

There are few practices to be considered in designing IoT solutions with AWS. If the right AWS services are used for customer requirements, then IoT solutions will be able to deliver results in a more secure, reliable, and scalable manner.

Design to operate at scale reliably

IoT systems must handle high-velocity and high-volume data captured by devices and gateways. The overflow of incoming data can be expected due to the sudden growth of the business, or sometimes due to a malicious attack. In such cases, the cloud system architecture should be scalable to handle such data.

The best approach is to send data to queue, buffer or real-time in-memory databases before storing it in storage. This helps to achieve real-time events and to slow down the data insertion rate to prevent the database crash or to prevent a slower response.

The device can publish data to AWS Kinesis, or AWS IoT rule can be used to forward data to AWS SQS and Kinesis to store it in time-series stores like AWS S3, Redshift, Data Lake, or Elastic search for data storage. These data store can be used to generate custom dashboard or AWS Quick Sight dashboards.

AWs IoT

Fig 1: AWS IoT- Prevent Data loss

Route large data volumes through data pipelines

Consuming incoming data from device topics directly to a single service prevents systems to achieve full scalability. Sometimes, such an approach limits the availability of the system on events of failure and data flood.

AWS IoT Rules Engine is designed to connect endpoints to AWS IoT Core in a scalable way. But all AWS services have different data flow properties and its own pros and cons. All services cannot be used as a single point of entry to the system. Sometimes it can create subsequent failure with no recovery. For example, in cases of high-volume data, consider buffering (Elasti Cache) or queuing (SQS) the incoming data before invoking other services, which enables the ability to recover from subsequent failures.

AWS IoT Rules Engine allows the triggering of multiple AWS services like Lambda, S3, Kinesis, SQS, SNS in parallel. Once data is captured by the IoT system, it then enables AWS endpoints (other AWS services) to process and transform data. This enables the ability to store data into multiple data stores simultaneously.

The most secure and best way to ensure all data is processed and stored is to redirect all device topics data to an SNS which is designed to handle data flood processing, ensuring that incoming-data is reliably maintained, processed and delivered to the proper channel. To make it more scalable, multiple SNS topics, SQS queue, Lambda for a different/group of AWS device topics can be used. One should consider storing the data in safe-storage like a Queue, Amazon Kinesis, Amazon S3, and Amazon Redshift before processing. This practice ensures no data loss due to message floods, un-wanted exception code or deployment issues.

AWS design

Fig 2: AWS IoT – Using Topics and Rules to redirect data to proper channel

Automate device provisioning and upgrades

As the business grows and numerous devices connect to the IoT ecosystem, manual processes such as device provisioning, bootstrapping the software, security configuration, rule-actions setup, device OTA upgrades – are not feasible. Minimizing human interaction in the initialization process and upgrades is important to save time and cost.

Designing built-in capabilities within the device for automated provisioning and leveraging the proper tools that AWS provides to handle device provisioning and management, allows systems to achieve desired operational efficiencies with minimal human intervention.

AWS IoT provides a set of functionalities which can be used for batch import with a set of policies that can be integrated with dashboard or manufacturing process where a device can be pre-registered to AWS IoT and certificates can be installed in the device. Later, device provisioning flow can claim device and attach with user or any other entity. AWS provides the facility to trigger and track OTA upgrades for devices.

Adopt scalable architecture for custom components

As IoT system connects to external world devices, the scope doesn’t end by connecting, controlling and reporting of devices. Think about adopting the latest technologies like Data Science and Machine Learning or integrating third-party components in IoT system like IFTTT, Alexa or Google Home. The Architecture of IoT should ensure that the external components can be easily integrated into solution without any performance bottlenecks.

Check for offline access and processing

Sometimes it is not necessary to process all your machine data in the cloud. In many cases, there is no continuous internet connectivity available. For such a scenario, add AWS Greengrass at the edge. Greengrass processes and filters data locally on edge and reduce the need to send all device data upstream. One can capture all data, hold it for a limited amount of time and send it to cloud on error events or on demand/request. If there is a need for time-series data, then one can schedule a periodic process that sends device data to the cloud which can be used for future enhancements like AWS Machine Learning models and cloud analytics tools.

AWS Greengrass

Fig 3: AWS Greengrass – Adding processing on Edge

Choose right data storage IoT systems generate high-speed, high-volume, and varieties of data. Each IoT device or device topic can have different formats, which may not be manageable through a single database or a similar type of data-store. An architect should be careful while choosing database format and data-store. Sometimes single data-store works fine, or hybrid data-store for a different purpose helps to achieve high throughput. Frequently used static data can be stored in the Elastic ache which helps to improve performance. Such practices help to achieve scalability and maintainability of the system.

Filter and transform data before processing

All incoming data to the IoT system may require processing or transforming, after which it can be redirected to storage as it is. AWS IoT rule provides action to redirect messages to different AWS services. An architect should dive all data in different forms i.e. processing-needed, ignored/static data (like Config) and direct storage.

AWS IoT service Architecture

IoT Application Integrations

Fig 4: AWS IoT – Put all data together

AWS IoT helps to achieve quick device connectivity, secure data ingesting, easy device management, multi-protocol support and much more. IoT solution enabler uses above best practices to deliver solutions across multiple domains like logistics, healthcare, Home Automation, Security and Surveillance, etc.

By Chandani Patel

Chandani Patel

Chandani is AWS Certified Solution Architect, AWS Business & Technical Professional, Technical Lead on several domains – Cloud Solutions, IoT Solutions, ML&Data Science. She is Cloud Solution Architect with expertise in designing, developing and architecting cloud solutions for public clouds (Azure, AWS, Google & Bluemix), private clouds & hybrid clouds.

View Website
Small Business Security

Ransomware, Backups and the Aging IT Specialist

Small Business Security Right now, two technology trends characterize the small business ecosystem: a growing migration to the cloud, and a growing susceptibility to cybercrime, ...
Big Commerce

Ecommerce and Social Media: Shopping In The Cloud

Ecommerce and Social Media By 2018, retail ecommerce will reach nearly $2.5 trillion worldwide, according to eMarketer’s forecasts. Currently, China tops the ecommerce markets, followed ...
Vibhav Agarwal

Cloud And Cybersecurity: 5 Things CISOs Need To Consider

The Cloud and Cybersecurity Tomorrow’s digital enterprise is at war today. War not only with external cybersecurity hackers and viruses, but also within the organization ...
ARS

Surprise! Uber and Lyft don’t like NYC’s new ride-hail rules

/
Uber CEO Dara Khosrowshahi, who took the helm of the controversial company back in 2017, is known for being pretty unflappable. He was even upbeat during the company’s second quarter earnings ...
Cisco News

Managing your SAP Digital Transformation Journey

/
Digital Transformation. We’ve heard the words, but have you wondered what it is all about? Digital Transformation is a strategic directive to redefine your business practices and processes to gain ...
Reuters news

Exclusive: U.S. set to give Huawei another 90 days to buy from American suppliers – sources

/
SINGAPORE/WASHINGTON (Reuters) - The U.S. Commerce Department is expected to extend a reprieve given to Huawei Technologies that permits the Chinese firm to buy supplies from U.S. companies so that ...