This objective of the project was to collect, process, and monitor live satellite data. The data engineering process includes extracting, transforming, and loading satellite data from Event Generator into Delta Lake. Process batch data from the data lake to the delta lake.
We built a dashboard in PowerBI for aggregated data from Delta Lake, build the databrick SQL alerts, and created rules engine and rules notification engine for customized complex rules from users for the notification alert.
•Businesses will be able to monitor the live satellite data and make decisions accordingly.
•Data owners will be able to monitor the data based on alerts and will be notified if there is any abnormality.
•Businesses will be able to create customized, complex rules for their data table to get notified if any abnormality happens.