Devapt project is deprecated, no security fix and no updates.
Welcome on Devapt project documentation (v2 or more)
With Devapt you can easily develop powerfull application.
Devapt libraries are usefull to quickly develop applications with more JSON configuration and less code.
What is Devapt ?
The Developpers Application Toolkit (Devapt) is a usefull package for developers: create quickly and securely a complex distributed set of applications or a simple alone application into one or more nodejs.
This project is based on Javascript (ECMAscript 6 transpiled in ES5) and nodejs. Devapt contains a server runtime and a set of client features. Devapt help you to write less code and to use easily more features.
The principle is simple: you write some configuration files for distributed nodes, models, views and menus and the you start the application. The framework will automatically generates the server and manages common features (logges, traces, metrics, restful, security...). Simply launch index.html and your application is up a rich user interface and many features.
Built-in features cover many important subjects as: authentication, restfull, http server, distributed services, logs, metrics... Main characteristics of DEVAPT architecture is dynamic behaviors, reactive programming, object oriented ES6 javascript, gulp build chain All of this on NodeJS.
My main initial purpose was to provide me a simple way to play with new scientifical or computer concepts by writing some labs applications.
Where to find Devapt ?
Devapt is a set of core libraries to help producing applications:
Project | GitHub | API Manual | NPM | Demo | Issues |
---|---|---|---|---|---|
devapt-core-common | Repo | Manual | NPM | Demo | Issues |
devapt-core-browser | Repo | Manual | NPM | Demo | Issues |
devapt-core-server | Repo | Manual | NPM | Demo | Issues |
devapt-core-services | Repo | Manual | NPM | Demo | Issues |
Others libraries provide usefull features.
Project | GitHub | API Manual | NPM | Demo | Issues |
---|---|---|---|---|---|
devapt-features-foundation6 | Repo | Todo | NPM | Demo | Issues |
Applications are good examples:
Project | GitHub | API Manual | NPM | Demo | Issues |
---|---|---|---|---|---|
devapt-app-testbus | Repo | Todo | NPM | Demo | Issues |
How does Devapt work ?
With Devapt you define:
- A topology, simple (one application, one server, one service) or complex (many services distributed for many applications on many distributed servers).
- Some resources (models, views, menubars, menus)
- Security rules
A topology contains:
- nodes: A node is a nodejs process with an unique name. You can have many nodes on the same machine. Each node communicates with other node through messaging buses (messages, metrics, logs).
A node can have one or many servers.
- servers: A server provides one or more services and listen client connections through a couple (host, port).
A server has a unique name and can be one of types: restify, express, socket.io (coming soon)...
- services: A service offers one feature to clients. A service can be one of: static assets providers, generic middleware provider, RESTfull provider, resources provider...
- applications: An application contains some services and is provided through nodes and servers.
An application can have one service on one server on one node. Or many services on many servers on many nodes.
An application contains:
- modules: functional features for applications (a set of preconfigured UI for example).
- plugins: technical fearures for applications (a rendering provider for example).
- security: defines authentication and authorization rules.
The rendering engines are plugins and rendering classes are stateless: state is stored in a Redux store. See Getting Started with Redux for flow concepts.
Devapt is a smart glue between many powerfull projects
Thanks for all projects leaders and contributers.
The given list is an extract of all used or inspired projects.
Main dependancies:
Servers and datas access:
- Express: http://expressjs.com/
- Restify: http://restify.com/
- Epilogue: https://github.com/dchester/epilogue
- Sequelize: http://sequelizejs.com
- Socket.io: http://socket.io/
Security:
Foundations:
- BaconJS: https://baconjs.github.io/
- SimpleBus: https://github.com/ajlopez/SimpleBus
- jQuery: https://jquery.com
- Immutable: https://facebook.gith)ub.io/immutable-js/docs/#/
- Redux: http://redux.js.org/docs/introduction/Motivation.html
- Moment: http://momentjs.com/
- Vantage: https://github.com/dthree/vantage
- Bunyan: https://github.com/trentm/node-bunyan
- Winston: https://github.com/winstonjs/winston
- Circuit breaker: https://www.npmjs.com/package/circuit-breaker or https://github.com/yammer/circuit-breaker-js
- LowDb: https://github.com/typicode/lowdb
Rendering:
- Foundation by ZURB: http://foundation.zurb.com/sites.html
- Mustache: https://github.com/janl/mustache.js
- React (coming soon): http://facebook.github.io/react/docs/tutorial.html
Dev and build dependancies: (coming soon.)
- Babel: https://babeljs.io
- Chai: http://chaijs.com/
- Gulp: http://gulpjs.com/
- Mocha: https://mochajs.org/
- JSDoc: http://usejsdoc.org/
Version
Current active version is 2.x.x
Versionning use standard method: M.m.p with M is a major change, m a minor change, p a patch.
License
Copyright Luc BORIES 2015-2016
Apache Version 2 license.
Please enjoy using devapt projects and let me know if they are usefull for you.
Overview
With DEVAPT you can easily develop powerfull application. Built-in features cover many important subjects as: authentication, restfull, http server, distributed services, logs, metrics... Main characteristics of DEVAPT architecture is dynamic behaviors, reactive programming, object oriented ES6 javascript, gulp build chain All of this on NodeJS.
Projects structure
All devapt-* github repositories have the same structure: a master branch with the last tagged tree, a develop branch with latest comited updates and tags.
All devapt-* projects have a common folders tree: project root\
- bower_components (only for browser features)
- build: building scripts
- dist: build codes and resources
- dist/js: build javascript codes
- dist/resources: build resources configuration
- node_modules: nodes packages
- public: shared assets for browsers
- public/app_name/css: CSS assets
- public/app_name/html: HTML assets
- public/app_name/img: images assets
- public/app_name/js: javascript assets
- public/favico.png: browser project icon
- src: project sources
- src/js: javascript sources
- src/resources: resources configuration sources
- tmp: temporary application files (logs)
- bower.json: browser packages configuration
- gulpfile.js: main build script
- LICENSE: project license
- package.json: nodes packages configuration
- README.md: project description
Some differences appears between library and application folder.
Versionning use standard method: M.m.p with M is a major change, m a minor change, p a patch.
Development
git clone git://github.com/lucbories/devapt-core-common.git
cd devapt-core-common
npm install
npm test
Same commands for devapt-core-browser, devapt-core-server, devapt-core-services
Contribution
Feel free to contribute, you're welcome.
Check issues and solve it:
Submit your work:
If you submit a pull request, please be sure to add or update corresponding
test cases, and ensure that npm test
continues to pass.
Installation
Via npm on Node:
cd my_project/
npm init
npm install devapt-core-common --save
npm install devapt-core-browser --save
npm install devapt-core-serices --save
npm install devapt-core-server --save
To install and launch TestBus applicaton :
cd apps/
git clone git://github.com/lucbories/devapt-app-testbus.git
cd devapt-app-testbus
npm install --save-prod
or yarn install -- production
npm run nodeA
or yarn run nodeA
or
npm run nodeA
npm run nodeB
or
npm run nodeA
npm run nodeB
npm run nodeC
open a browser on http://my_domain:8080/assets/html/index.html
Usage
To install a devapt application, follow:
cd apps/
git clone git://github.com/lucbories/devapt-app-testbus.git
cd devapt-app-testbus
npm install --save-prod
To create an empty application from scratch (not recommended for devapt newbies):
cd my_app
npm init
npm install devapt-core-common --save
npm install devapt-core-browser --save
npm install devapt-core-services --save
npm install devapt-core-server --save
Create folders tree:
-build
-dist
-public
-public/assets
-public/assets/css
-public/assets/html
-public/assets/img
-public/assets/js
-src/js
-src/resources
-tmp
Create your first Devapt based HTML page in public/assets/html/mypage.html:
<html>
<head>
<meta charSet="utf8"/>
<title>Devapt TestBus Home</title>
<link rel="icon" type="image/png" href="/assets/img/favico.png"/>
<link type='text/css' rel="stylesheet" href="/assets/css/normalize.css" media="all" />
<style type='text/css'>
... your styles
</style>
</head>
<body>
<header>
<center>
<h1>MY FIRST APP</h1>
<h8><a href="/assets/html/mypage.html">Home</a></h8>
</center>
</header>
<hr>
<script>window.__INITIAL_STATE__ = {
"credentials":{
"tenant":"tenantA",
"env":"dev",
"application":"testbus",
"token":"",
"user_name":"",
"user_pass_digest":"",
"ts_login":0,
"ts_expiration":0,
"errors_count":0,
"renew_count":0,
"hash":undefined
},
"commands":{}
}
</script>
<div id="content">
... your HTML code
</div>
<script type="text/javascript" src="/socket.io/socket.io.js"></script>
<script type="text/javascript" src="/assets/js/vendor/browser.min.js"></script>
<script type="text/javascript" src="/assets/js/devapt-core-browser.js"></script>
<script type="text/javascript" src="/assets/js/devapt-bootstrap.js"></script>
<script type="text/javascript" src="/assets/js/app.js"></script>
</body>
</html>
Now you can write your Javascript code in public/assets/js/app.js:
window.devapt().on_runtime_created(
function()
{
console.log('runtime ready for application script')
write your code here: devapt init is finished and you can use Devapt function:
window.devapt(): Devapt bootstrap root with some helpers
window.devapt().runtime():main Devapt client runtime
window.devapt().ui(): main Devapt user interface manager
Example: get client session unique id
var session_uid = window.devapt().runtime().get_session_uid()
Example: ping master node
var target_name = 'master' (target is a node or a node part, node is server host or remote host or remote browser session)
var ping_svc_cfg = undefined (reserved for later)
var ping_svc_promise = window.devapt().get_service('pingpong', ping_svc_cfg)
var on_response_callback = function() {
console.log('pong received')
}
window.devapt().request_service(ping_svc_promise, 'devapt-ping', target_name, on_response_callback)
}
)
Devapt use configuration files to discover and build a runtime of dynamic resources.
First we declare our unique node called "nodeA", this node is the topology master and the application provider: File: dist/resources/nodes/nodeA.json
{
"is_master":true,
"name":"NodeA",
"master":{
"name":"NodeA",
"msg_bus":{
"package":"socketio",
"protocole":"ws",
"type":"Server",
"host":"localhost",
"port":5000
},
"logs_bus":{
"package":"socketio",
"protocole":"ws",
"type":"Server",
"host":"localhost",
"port":5001
},
"metrics_bus":{
"package":"socketio",
"protocole":"ws",
"type":"Server",
"host":"localhost",
"port":5002
}
},
"settings_provider": {
"source":"local_file",
"relative_path":"world.json"
}
}
Secondly we describe our application features topology: File: dist/resources/world/world.json
{
"nodes":"nodes":{
"NodeA":{
"host":"localhost",
"is_master":true,
"servers":{
"NodeALocal8080":{
"type":"express",
"port":8080,
"protocole":"http",
"middlewares":[],
"use_socketio":true,
"security": {
"authentication": {
"enabled":false,
"plugins":[]
}
}
},
"NodeALocalLogs":{
"type":"logs",
"port":"N/A",
"protocole":"bus",
"middlewares":[],
"use_socketio":false,
"security": {
"authentication": {
"enabled":false,
"plugins":[]
}
}
}
}
}
},
"tenants":{
"tenantA":{
"applications":{
"assets":{
"title":"Assets provider",
"url":"assets",
"license":"APACHE-LICENSE-2.0",
"provided_services":[],
"used_services":[],
"used_packages": ["my_app_pkg"],
"used_plugins":["DefaultRendering"],
"assets":{
"css":[],
"js":[],
"img":[],
"index":""
}
}
},
"packages":{
"my_app_pkg":{
"base_dir":null,
"resources":[],
"templates":[],
"includes":[],
"commands":[],
"services":{
"pingpong":{
"type":"ping_svc"
},
"html_assets":{
"type":"html_assets_svc",
"routes":[
{
"route":"/",
"directory":"../../public/assets/",
"required_prefixes":[],
"required_suffixes":[".js", ".js.map", ".css", ".html", ".png", ".jpg"]
}
],
"security":{
"enabled":false,
"authentication":{
"enabled":false,
"engines":[]
},
"authorization":{
"enabled":false,
"engines":[],
"permissions":{
"*":["*"]
}
}
}
}
}
}
},
"security":{}
}
},
"deployments":{
"tenantA":{
"assets":{
"assets":{},
"services":{
"topology":{ "servers":["NodeALocal8080"], "filters":["*"] },
"html_assets":{ "servers":["NodeALocal8080"], "filters":["*"] },
"pingpong":{ "servers":["NodeALocal8080"], "filters":["*"] }
}
}
}
},
"security":{
"is_readonly":false,
"datasources":[],
"authentication": {
"enabled":false,
"plugins":{},
"default_plugins": []
},
"authorization": {
"enabled":false
}
},
"plugins":{
"DefaultRendering":{
"type":"rendering",
"file":"../../../node_modules/devapt-core-common/dist/js/default_plugins/rendering_default_plugin"
},
"DefaultServices":{
"type":"services",
"file":"../../../node_modules/devapt-core-services/dist/js/default_plugins/services_default_plugin"
}
},
"loggers":{},
"traces":{}
}
Launch your runtime nodes and servers:
cd my_app
node node_modules/devapt-core-server/dist/js/start_node.js -n NodeA -t dist/resources/world -b node_modules/devapt-core-server/dist/js -s NodeA:NodeALocal8081
Example with settings in many files:
"resources//worldworld.json":
{
"nodes":"nodes.json",
"tenants":{
"tenantA":{
"applications":"applications.json",
"packages":"tenantA_packages.json",
"security":{}
}
},
"deployments":"deployments.json",
"security":"security.json",
"plugins":"plugins.json",
"loggers":"loggers.json",
"traces":"traces.json"
}
"nodes.json":
{
"nodes":{ same settings than above }
}
"services.json":
{
"services":{ same settings than above }
}
Repeat on each files on the same manner.
Design
Devapt is full stack solution. So it's usefull to provide an integrated framework to build your application with configuration files. On the other hands it's difficult to have an overview of all parts and undestand how all this parts work together.
First, devapt is provided throught specialized core projects and addon features. If you write an application you don't need to know all details. But if you want to add features, you should understand on what starting to build your feature. Details of projects
Main devapt concepts are:
- Node: each application can be distributed on one or many nodes. Each node run on a NodeJS instance with some inter node communication channels. One or many nodes can run on one NodeJS instance. One or many NodeJS instance can run on one hostnames.
- Node settings: JSON file which describe one or many nodes with their names, security, debug trace and communication channels.
- Application settings or configuration or description: main JSON file which define application features, packages, resources (UI components...), debug traces and security.
- Bus: all nodes communicate together with messages throught a communication bus. An application can have one or more buses, one for metrics, one for logs and one between components.
- Tenant: a tenant is a set of applications and packages with its own security access. for example, we can have one tenant per company on a shared server (SaaS).
- Service: a service provide a feature which responds some datas on a request. For example, a ping pong service responds a 'pong' on each 'ping' request.
- Server: a server (not a hostname) is an engine which provides services and others features. For example, a logs server exists to process received logs.
- Metric: metrics are collected datas from a node, an application, a server, a service... which help to monitors system health.
- Rendering: it's the process of producing DOM content for the browser page.
- Virtual DOM: it's a technology which can produce DOM content outside of browser and to apply differential changes on a page. It enhance performance and decrease browser load.
- Security: security is an important need for all usages. Security is everywhere in devapt apps but all parts aren't finished yet. (TODO)
How an application work
An application has one directory but two targets: server and browser. On server side, an application launches nodes and services. On browser side, the page receive its content from the server: it's a lightweight HTML page with two scripts: a credentials record and an initial application state. Each user action on the interface call some scripts and request server if needed.
Features
Examples
Define two routes to serve static files:
- http://myserver/ with files read from an absolute path /.../devapt-app-devtools/dist/assets
- http://myserver/myapp/css/*.css with files read from a project relative path ./public/.
"html_assets_1":{
"type":"html_assets",
"routes":[
{
"is_global":true,
"route":"/",
"directory":"/.../devapt-app-devtools/dist/assets",
"default":"index.html"
},
{
"route":"/css/.*.css",
"directory":"./public"
}
]
}
Tutorials
Some usefull applications are:
- DEVTOOLS which actualy display settings, logs, buses messages, metrics and topology. DEVTOOLS app project
- TESTBUS wich helps to test distributed features: basic ping/pong communication, messaging, logs, topology. TESTBUS app project
- TESTUI wich helps to test user interface features. (coming soon) TESTUI app project
Projects map
Devapt projects refcard.
Development Map
Projects core dependencies (package.json)
Project | devapt-core-common | devapt-core-browser | devapt-core-services | devapt-core-server |
---|---|---|---|---|
devapt-app-testbus | x | x | x | x |
devapt-app-testui | x | x | x | x |
devapt-app-devtools | x | x | x | x |
devapt-core-doc | x | x | x | x |
devapt-core-common | ||||
devapt-core-browser | x | |||
devapt-core-services | x | |||
devapt-core-server | x | x | ||
devapt-features-foundation6 | x | |||
devapt-features-cytoscape | x | |||
devapt-features-sparklines | x | |||
devapt-features-labs | x | x | ||
devtools | x | x |
Projects features dependencies (package.json)
Project | devapt-features-foundation6 | devapt-features-cytoscape | devapt-features-sparklines | devapt-features-labs | devtools |
---|---|---|---|---|---|
devapt-app-testbus | |||||
devapt-app-testui | |||||
devapt-app-devtools | x | ||||
devapt-core-common | |||||
devapt-core-browser | |||||
devapt-core-services | |||||
devapt-core-server | |||||
devapt-features-labs | |||||
devapt-app-labs | x | x | |||
devtools | x | x | x |
devapt-core-doc
Purpose: generate core projects documentation (API, manuals, tutorials)
Devapt projects dependencies:
- devapt-core-browser
- devapt-core-common
- devapt-core-server
- devapt-core-services
devapt-core-common
Purpose: common features for all others projects.
Devapt projects dependencies:
- none
devapt-core-services
Purpose: services implementation for browser/server projects.
Devapt projects dependencies:
- devapt-core-common
Provided services:
- assets routes
- logs stream provider
- messages/logs/metrics stream provider and dispatching
- filtered metrics stream provider
- middleware based routes provider
- ping/pong provider
- resources configuration provider
- security login/signup/logout/... provider (coming soon)
- topology description provider
devapt-core-server
Purpose: server features. Provides assets, resources, services, security, plugins, server runtime and metrics.
Devapt projects dependencies:
- devapt-core-common
- devapt-core-services
devapt-core-browser
Purpose: browser features. Provides commands, client runtime and components.
Devapt projects dependencies:
- devapt-core-common
devapt-app-testbus
Purpose: messages bus testing application.
Devapt projects dependencies:
- devapt-core-common
- devapt-core-browser
- devapt-core-server
- devapt-core-services
devapt-app-testui
Purpose: user interface testing application.
Devapt projects dependencies:
- devapt-core-common
- devapt-core-browser
- devapt-core-server
- devapt-core-services
devapt-features-foundation6
DEVAPT FOUNDATION 6 features project
Purpose: Foundation 6 rendering features.
Devapt projects dependencies:
- devapt-core-common
devapt-features-cytoscape
DEVAPT CYTOSCAPE features project
Purpose: Cytoscape rendering features.
Devapt projects dependencies:
- devapt-core-common
- devapt-core-browser
devapt-features-sparklines
DEVAPT SPARKLINES features project
Purpose: Sparklines rendering features.
Devapt projects dependencies:
- devapt-core-common
- devapt-core-browser
devapt-features-labs
Purpose: labs features. Provides calculus and plots.
Devapt projects dependencies:
- devapt-core-common
- devapt-core-browser
devapt-app-labs
Purpose: Amazing labs application.
Devapt projects dependencies:
- devapt-core-common
- devapt-core-server
- devapt-core-services
- devapt-features-foundation6
- devapt-features-labs
devapt-devtools
Purpose: development tools features.
Devapt projects dependencies:
- devapt-core-common
- devapt-features-cytoscape
- devapt-features-foundation6
- devapt-features-sparklines
devapt-app-devtools
Purpose: development tools application.
Devapt projects dependencies:
- devapt-core-common
- devapt-core-server
- devapt-core-services
- devapt-devtools
- devapt-features-cytoscape
- devapt-features-foundation6
- devapt-features-sparklines
Roadmap
2.0.x breaking changes and new features (CURRENT)
MAIN GOALS
When I was working on devtools features with devapt 1.0.x, I meet some trouble with missing features. An other problem was to deal with topology definition and deployment information previously stored in many pieces of code. It was difficult to obtain a global view of defined or deployed topology. So I pull out deployment information from topology definition.
One missing feature for fine deployment was the lake of customer and version of defined items.
Main added features:
- isomorphic Credentials class to deal with session information on browser or on server (tenant, application, username, token...). (DONE)
- isomorphic data management classes: DataStore, DataRecord, DataCollection...
- topology management classes: TopologyDefineItem...(DONE)
- topology deployment classes: TopologyDeployLocalNode... (DONE)
- browser router (for single page application and more) (DONE)
- isomorphic cache management (DONE)
- versionned Collection to register versionned items (CURRENT)
Main breaking changes:
- settings json format has evolved
- module was renamed to package
- change Collection.$items from [] to {}, use lodash as collection backend
New topology tree is now:
world: the root of the topology
-nodes
--nodeA
---serverA1
---serverA2
-tenants
--tenantA
---applications
----applicationA
-----url
-----license
-----provided_services
-----used_services
-----used_packages
-----used_plugins
-----assets
---packages
----packageA
-----base_dir
-----resources
-----services
---security
-plugins
--pluginA
---type
---file or package
-deployments
--tenantA
---applicationA
----assets
----services
-security
-loggers
-traces
That's why previous roadmap is outdated, sorry for the changes.
enhance core features, architecture and stability (previous 1.1.x)
PROJECT
- refactor directories into common/, browser/, server/ (DONE)
- implement Unit tests for base classes (CURRENT)
- use babel-register to debug without build to dist/ directory (DONE)
- test on nodejs 4.4, 5.1, 6.x (DONE)
FEATURES
- implement browser router for single page application (DONE)
- enhance devapt-devtools: runtime registry viewing (DONE)
- enhance devapt-devtools: runtime topology viewing (DONE)
- enhance devapt-devtools: metrics on topology (TODO)
- enhance devapt-devtools: application Redux state viewing (TODO)
- enhance devapt-devtools: application Redux state history player (TODO)
- implement unit test for instances, traces, loggers... (CURRENT)
- implement topology registry (DONE)
- implement topology runtime (DONE)
- implement generic api for state storing (Redux, Map) (DONE)
- implement cache management classes for browser and server (DONE)
enhance datas access features (previous 1.3.x)
PROJECT
- document datas access layers (TODO)
- write Unit tests for datas access classes (TODO)
FEATURES
- enhance devapt-devtools: create datas access dashboard (TODO)
- enhance devapt-devtools: create datas admin backend (TODO)
- implement homogeneous datas access for all datasources (DONE)
- enhance sequelize datasource for SQL databases:MySQL, MariaDb, PostgreSQL, SQLite, MSSQL (DONE)
- implement waterline ORM layer (TODO)
- implement Redis datasource for caching, messaging, NoSQL database (TODO)
- implement datas management classes for browser and server (DONE)
- implement datas cache on distributed nodes (TODO)
2.1.x enhance distributed features and documentatioon (LATER)
PROJECT
- update web site on github with project/, technical/... (TODO)
- document build process (TODO)
FEATURES
- implement distributed services consumers (TODO)
- implement master configuration changes: receive and propagate complete/add/remove/update (TODO)
- implement master node revocation and promotion (TODO)
- implement node settings filter (TODO)
- implement swagger API player (TODO)
- implement circuit breaker on browser (TODO)
2.2.x enhance security features (LATER)
PROJECT
- document security process (TODO)
- write Unit tests for security classes (TODO)
FEATURES
- enhance devapt-devtools: create security dashboard (TODO)
- enhance devapt-devtools: create security admin backend (TODO)
- implement security routes /login, /signup, /logout (TODO)
- implement Passport security plugin (TODO)
- implement authorization with ACL (TODO)
- implement security token (TODO)
- implement registry setting validation with a json schema (TODO)
2.3.x implement Cloud features (LATER)
PROJECT
- document Cloud integration process (TODO)
- write Unit tests for Cloud integration classes (TODO)
FEATURES
- implement AWS DynamoDB datasource (TODO)
- implement AWS Lambda integration (TODO)
- implement AWS Security integration (TODO)