Administration and Logging
The Administration and Logging Module consists of these four parts:
- Logging Module (written in C++)
- Apache Cassandra Database
- REST Admin Module (written in JAVA)
- Web GUI
The Logging Module writes all messages that will appear in the system to a database. These messages will be shown on the Web GUI.
To run the Logging Module, you have to install the Cassandra C++ Driver, which could be found here:
Normal users have to download and install following two files:
- libuv_1.7.X-X_amd64.deb
- cassandra-cpp-driver_2.X.X-X_amd64.deb
To build the Logging Module it is necessary to install the development driver, too:
- cassandra-cpp-driver-dev_2.X.X-X_amd64.deb
Developers might also want to install the debug driver.
Once you have started (and enabled) the Logging Module, it will connect to the Apache Cassandra Database. After this, the module will set a subscription to the messages on the ZMF-Bus. For a better performance in large systems, you could change the subscription topic from all messages to a more specific one. Also you can use multiple instances to log a specific type of message on multiple hosts.
Below is the flow diagram of the Logging Module:
As mentioned before, we are using Apache Cassandra DB 2.0.X to store these messages. Cassandra is a high-performance NoSQL-Database. The query language is an SQL look alike called CQL. In a productive system - which creates high traffic - you should run Cassandra on multiple nodes (so called clusters). These nodes could be set at the config file. The config key is: ZSDN_LOGGING_MODULE_CASSANDRA_URL
A installation guide for Debian-based systems can be found here:
Note: to use CQL out of a module, you have to install the driver first!
Keyspace: logging Table: messages
| id | messageType | senderType | senderId | timeStamp |
|---|---|---|---|---|
| uuid | blob | varint | varint | timestamp |
To start Cassandra, execute following command from shell:
bin/cassandra -for, if you have used the link as mentioned before to install Cassandra:
service cassandra startTo get the cqlbash run:
bin/cqlshTo insert data in a fresh Cassandra DB execute the commands below:
CREATE KEYSPACE IF NOT EXISTS logging WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };```<br>```
CREATE TABLE IF NOT EXISTS logging.messages (id uuid, messageType blob, senderType varint, senderId varint, timeStamp timestamp, PRIMARY KEY (id, timeStamp));```<br>```
INSERT INTO logging.messages (id, messageType, senderType, senderId, timestamp) VALUES (uuid(), 0x010000000000000000000100, 1, 0, dateOf(now()));For more information, we recommend to read some tutorials at the Apache Cassandra Wiki:
Connect to Server via SSH
ssh user@serverTo start Cassandra:
sudo systemctl start cassandraShutdown Cassandra:
sudo service cassandra stopTo open cqlsh:
cqlshThe Java Administration Module provides a Web GUI and a REST interface which provides data and allows module control. It is a Java web application archive (WAR) which runs on the tomcat server or standalone with embedded tomcat server. The project itself is a Maven web project, which uses the Apache Maven build automation tool for the projects dependencies and build life cycles.
The WebAdmin is utilizing the following packages:
- javax.servlet
- JAX-RS: Java API for RESTful Web Services
- Jersey RESTful Web Services framework
JAX-RS provides the base of the REST interface in JAVA while Jersey is a framework which implements the JAX-RS API. The application itself is a servlet utilizing javax.servlet.
The following dependencies are included through the projects Maven pom file: asm, gson, cassandra-driver-core, servlet-api, rest-assured, json-schema-validator, junit.
As seen above, you don't have to install the Cassandra (Java) driver by yourself.
The WebAdmin can be deployed on a webserver or started as standalone jar with embedded Apache Tomcat 7. The maven build will build both, the war file and the standalone jar.
Once correctly started the Web GUI can be accessed at http://localhost:8080/
The REST interface is available under http://localhost:8080/rest
To start the WebAdmin module as standalone jar you can simply run the web-admin-standalone.jar. This will start the embedded tomcat server and deploy the WebAdmin application automatically.
The Web Admin Module is available as WAR file and can be deployed on a tomcat server. It has been used and tested on tomcat 7 but you might be able to run it on a different version. The WebAdmin Module should be deployed as root application (e.g. by naming it ROOT.war and deploy).
The Web Admin Module provides a REST Interface. If the user doesn’t want to use the implemented front end, the user can create his own and use the REST Interface of the REST Admin Module.
GET requests:
Name: Logging
URL: rest/messages
Result: All logging messages
Name: Logging filtered by starttime and endtime
URL: rest/messages/filtered?starttime={time}&endtime={endtime}
Result: Messages filtered by time
Name: Module registry
URL: rest/modules/
Result: All modules with status
Name: Single Module registry
URL: rest/modules?id={ModuleID}
Result: Module with status
Name: Topology
URL: rest/topo
Result: Topology as JSON
PUT requests:
Name: StartStop
URL: rest/modules/control?id={ModuleID}&action={enable/disable/stop}
Therefore the REST Interface itself can be separated into the following three classes: RESTModules, RESTMessages, RESTTopology. The main application logic itself is inside the AdminModule class, which extends the JMF AbstractModule class and allows access to the functions of the JMF. An instance managing class called ZMFmanager controls the creation and initialization of the AdminModule, as the application is a servlet and cannot be started in the regular way (main method).
In the Web Admin Module you will find an example implementation for a web UI. To get access, just run the Web Admin Module and open the URL / in your Browser.
The Web UI is written in HTML5 and JavaScript. AngularJS is used to create the tables and send http requests. These requests will be send to the REST Interface of the Java Administration Module. D3.js is used to create the topo tree.
The web UI has three main functions:
- Logging
- Module Administration
- Topology View
At the Logging tab, the user can look at the messages that will be send through the ZMF as explained before. The first field is the Message Type this explains what message that specific message is. The second field is the Sender Type, which describes the kind of module that has published this message. The third field is the Sender ID so you could see which module the message has published. In the last field you will find the timestamp of the message.
The data in the response is a JSON object which has the following structure:
JSON structure for Logging
{
{
id : '',
messageType : '',
senderId : ,
senderType : ,
timeStamp : ''
}, {
id : '48bc...',
messageType : 'foo',
senderId : 12,
senderType : 1,
timeStamp : '2015-07-14 15:00:00+0200'
}
}
According to our module life cycle, not every module that is started once is also enabled. The Module Administration can give you an overview, which module is started and/or enabled. If a module is only started - but not enabled - the user can enable the module here. Also the user can disable a module or stop it. Please notice: you can't restart a module that is stopped via the Web GUI.
JSON structure for Module Administration
[
{
"name" : "",
"version" : ,
"UniqueId" : ":",
"currentState" : ""
}, {
"name" : "LinkDiscoveryModule",
"version" : 0,
"UniqueId" : "5:0",
"currentState" : "Active"
}
]
The Topology View will allow you a global view on the network. The links between the nodes will change the color from black to red, depending on the load.
JSON structure for Topology View
{
"nodes" : [{
"name" : "",
"id" : ""
}, {
"name" : "node 1",
"id" : "unique_id_1"
}
],
"links" : [{
"source" : "",
"target" : "",
"value" :
}, {
"source" : "unique_id_1",
"target" : "unique_id_2",
"value" : 123
}, {
...
}
]
}