Create Custom Data Flows
This Bundle consists of three key components to start using our predefined dataflow of to start experimenting and create your own dataflows.
The Datastore used for this Bundle is MongoDB. This bundle had a specfic Database on MongoDB(Tengu-DB) which will be used to store all the data gathered from this dataflow. The Database is only accesible by using the provided Bundle-API. This API contains GET-calls to gather data from the Database.
Our predefined dataflow will gather information from a Messagebroker, ActiveMQ, where there is one topic(Tengu-Topic) which will be used to subscribe and send messages to. The messages that are received on the Tengu-Topic come from an RSS-feed of StackOverflow posts that have the following tags:
Java, Python, ActiveMQ or MongoDB. The dataflow the subscribes to this specific topic and has a relation with the specific topic instead of with the whole service.
The Admin Panel of ActiveMQ is available at
http://x.x.x.x:8161 and uses basic auth to login. To Password is generated during the installation and will be visible in de status of the service.
The Node-RED service will make it possible to easily create new dataflows or tweak the current predefined dataflow. To Deploy a certain dataflow on Node-Red we use a Node-RED deployer. This makes it possible to add as much workflows as desired. In that case the Node-RED deployer and the Node-RED service do not need to know all the other services that might be necessary to deploy a certain workflow. The Node-RED Deployer will install the required nodes that are needed to deploy a specific workflow.
In the Node-RED interface you can enable or disable debugging and logging on the current dataflow or you can create your own dataflow with data from thew Tengu-Topic or from another datasource like Twitter, RSS-feed...
this bundle can be deployed in your juju environment by cloning this repo and clone the required charms that are not avilable on the juju charm store. to deploy this bundle use:
juju deploy cs:~tengu-team/bundle/create-custom-data-Flows-3
- Your own Big Data workspace
- Use all Juju-charmed technologies
- Machine Learning, IoT, Microservices
Embed this bundle
Add this card to your website by copying the code below. Learn more.