DENIM Evolving

📣 Description

This application enables to get evolutionary insights of a microservices architecture from a data perspective.

⭐ Features

Here is a summary of the features currently supported.

Metrics

Description

It computes some metrics based on a static and/or dynamic analysis report of a microservices architecture.

Implementation status

Metric	Implementation status
Locations list	🌕
Technologies list	🌕
Operations list	🌕
Methods list	🌕
Concepts list	🌕
Directories number	🌕
Files number	🌕
Code fragments number	🌕
Concepts number	🌕
Operations of technologies breakdown	🌕
Concept breakdown	🌕
Lines of Code (LoC)	🌕
Technology Inclusion Rate (TIR)	🌕
Concept Inclusion Rate (CIR)	🌕

How to?

INPUT

Invoke the following routes by sending a static and/or dynamic analysis report in JSON (obtained from DENIM Reverse Engineering) inside the request.

Routes:

• POST /list/locations returns the list of locations declared in the microservices architecture.
• POST /list/technologies returns the list of technologies used in the microservices architecture.
• POST /list/operations returns the list of operations used in the microservices architecture.
• POST /list/methods returns the list of methods used in the microservices architecture.
• POST /list/concepts returns the list of data concepts used in the microservices architecture.
• POST /metric/number/directories returns the number of directories in the microservice architecture.
• POST /metric/number/files returns the number of files in the microservice architecture.
• POST /metric/number/code-fragments returns the number of code-fragments in the microservice architecture.
• POST /metric/number/concepts returns the number of concepts in the microservice architecture.
• POST /metric/breakdown/operations-technologies returns the static and/or dynamic breakdown of the operations in each technology (i.e., the frequency of each operation in each technology) in the microservices architecture.
• POST /metric/breakdown/concepts returns the breakdown of the concepts (i.e., the frequency of each concept) used in the microservices architecture.
• POST /metric/loc returns the LoC (Lines of Code, i.e. the total number of lines of code) of the microservices architecture.
• POST /metric/tir returns the TIR (Technology Inclusion Rate, i.e. the percentage of a files containing the technology) of the microservices architecture.
• POST /metric/cir returns the CIR (Concept Inclusion Rate, i.e. the percentage of files containing the concept) of the concepts used in the microservices architecture.

OUTPUT

Consult the response view object in JSON:

// POST /list/locations

e.g.,

[
  "https://github.com/<user>/<repository>/.../",
  "https://github.com/<user>/<repository>/.../<file path>.js",
  "https://github.com/<user>/<repository>/.../<file path>.js#Lx1Cx1-Lx2y2",
  ...
]
        
// POST /list/technologies

e.g.,

["javascript-api-express-call", "javascript-db-mongo-call", "javascript-db-redis-call"]

// POST /list/operations

e.g.,

["READ"]
        
// POST /list/methods

e.g.,

["get", "find", ...]

// POST /list/concepts

e.g.,

["concept1", "concept2", ...]

// POST /metric/number/directories

e.g.,

1

// POST /metric/number/files

e.g.,

1

// POST /metric/number/code-fragments

e.g.,

1

// POST /metric/number/concepts

e.g.,

1

// POST /metric/breakdown/operations-technologies

e.g., 

{
  "static": {
    "javascript-api-express-call": {
      "CREATE": 1,
      "READ": 2,
      "UPDATE": 3,
      "DELETE": 4,
      "OTHER": 0,
      "TOTAL": 10
    },
    "javascript-db-mongo-call": {
      "CREATE": 1,
      "READ": 2,
      "UPDATE": 3,
      "DELETE": 4,
      "OTHER": 0,
      "TOTAL": 10
    },
    "javascript-db-redis-call": {
      "CREATE": 1,
      "READ": 2,
      "UPDATE": 3,
      "DELETE": 4,
      "OTHER": 0,
      "TOTAL": 10
    }
  },
  "dynamic": {
    "javascript-api-express-call": {
      "CREATE": 1,
      "READ": 2,
      "UPDATE": 3,
      "DELETE": 4,
      "OTHER": 0,
      "TOTAL": 10
    },
    "javascript-db-mongo-call": {
      "CREATE": 1,
      "READ": 2,
      "UPDATE": 3,
      "DELETE": 4,
      "OTHER": 0,
      "TOTAL": 10
    },
    "javascript-db-redis-call": {
      "CREATE": 1,
      "READ": 2,
      "UPDATE": 3,
      "DELETE": 4,
      "OTHER": 0,
      "TOTAL": 10
    }
  }
}

// POST /metric/breakdown/concepts

e.g.,

{
    "concept1": 1,
    "concept2": 2,
    "concept3": 3,
    "concept4": 4,
    "TOTAL": 10
}

// POST /metric/loc

e.g.,

999

// POST /metric/tir

e.g.,

{
  "javascript-api-express-call": 0.5,
  "javascript-db-mongo-call": 0.00,
  "javascript-db-redis-call": 0.99
}

// POST /metric/cir

e.g.,

{
  "concept1": 1,
  "concept2": 0.5,
  "concept3": 0.99,
  "concept4": 0.01
}

Analyses

Description

It reflects on a static analysis report to derives some conclusions about the given codebase and its versions of a microservice architecture.

Implementation status

Analysis	Implementation status
Intra-version conceptual similarity coupling	🌕
Intra/Inter-version code fragment similarity coupling	🌕
Inter-version difference evolutionary coupling	🌕
Extra-version versional evolutionary coupling	🌕
Extra-version conceptual evolutionary coupling	🌕

How to?

INPUT

Invoke the following routes by sending a static analysis report in JSON (obtained from DENIM Reverse Engineering) inside the request.

Routes:

• POST /coupling/similarity/conceptual/ returns the matrix of conceptual coupling representing the frequency with which the top 10 concepts appear together in code fragments.
• POST /coupling/similarity/code-fragment/path/:path/technology/:technology/operation/:operation/method/:method/concepts/:concepts returns the code fragment in the microservices architecture that corresponds the given criteria.
• POST /coupling/evolutionary/difference returns the difference of repositories, directories, files, and code fragments between two versions of the microservices architecture.
• POST /coupling/evolutionary/versional/path/:path returns the files of the microservices architecture, along with the corresponding frequency, that were co-modified at the same time in the Git history.
• POST /coupling/evolutionary/conceptual/path/:path returns the concepts, along with the corresponding frequency, contained in files of the microservices architecture that were co-modified at the same time in the Git history.

ℹ️ Beware that the extra-version (evolutionary coupling) could require a valid GitHub API token (i.e., in order to extend the GitHub's API rate limits) in the .env file as follows:

GITHUB_API_TOKEN=github_...

⚠️ Beware that the extra-version evolutionary versional or conceptual coupling may consume a large number of GitHub API requests, which could take a while and decrease significantly the quotas.

Generate it from an active GitHub accounts. GitHub > Settings > Developer Settings > Personal access tokens > Generate new token > Enter a "Token name" > Define an "Expiration" date > Select "Public Repositories (read-only)" > Click on "Generate token". More information: Managing your personal access tokens

OUTPUT

Consult the response view object in JSON:

// POST /coupling/similarity/conceptual

e.g.,

{
  "concepts": ["concept1", "concept2", "concept3"],
  "matrix": [ [3, 1, 2], [1, 4, 3], [2, 3, 5] ]
}

// POST /coupling/similarity/code-fragment/path/:path/technology/:technology/operation/:operation/method/:method/concepts/:concepts

e.g., 

[
  {
    "location": "https://github.com/<user>/<repository>/.../<file path>.js#Lx1Cx1-Lx2y2",
    "technology": {
      "name": "<technology>"
    },
    "operation": {
      "name": "<operation>"
    },
    "method": {
      "name": "<method>"
    },
    "sample": {
      "content": "<sample>"
    },
    "concepts": [
      {
        "name": "<concept>"
      }
    ],
    "heuristics": "<heuristics>",
    "score": "<score>"
  }
]
        
// POST /coupling/evolutionary/difference

e.g.,

{
  "unchanged": [
    // ...
  ],
  "modified": [
    // ...
  ],
  "removed": [
    // ...
  ],
  "added": [
    // ...
  ],
}


// POST /coupling/evolutionary/versional/path/<path>

e.g., 

{
  "path": "https://www.github.com/<user>/<repository>/.../<path 1>.js#Ly1Cx1Ly2Cx2",
  "commits": 15,
  "files": {
    "https://www.github.com/<user>/<repository>/.../<path 1>.js#Ly1Cx1Ly2Cx2": 15,
    "https://www.github.com/<user>/<repository>/.../<path 2>.js#Ly1Cx1Ly2Cx2": 10,
    "https://www.github.com/<user>/<repository>/.../<path 3>.js#Ly1Cx1Ly2Cx2": 7,
    // ...
    "https://www.github.com/<user>/<repository>/.../<path x>.js#Ly1Cx1Ly2Cx2": 1
  }
}

// POST /coupling/evolutionary/conceptual/path/<path>

e.g.,

{
  "concept1": 10,
  "concept2": 5,
  "concept3": 3,
  // ...,
  "conceptX": 1
}

👩‍💻 Development details

Setup

See INSTALL file.

Test the app (manually)

Manual test suites are set up thanks through the Postman tool.

The tests are specified in the /test/manual directory and are named following the *.test.js pattern.

Test the app (unit testing)

Unit test suites are set up thanks to the Jest framework.

The tests are specified in the /test/unit directory and are named following the *.test.js pattern.

The configuration of Jest is stated in the /package.json file.

The tests running computes the code coverage.

Launching the tests

• Launch the unit tests.

  npm run test_unit

Test the app (integration testing)

Integration test suites are set up thanks to the SuperTest framework.

The tests are specified in the /test/integration directory and are named following the *.test.js pattern.

The configuration of Jest is stated in the /package.json file.

Preparing the environment with Docker

• Launch the application on Docker (cf. Dockerize the application).

Launching the tests

• Launching integration tests.

  npm run test_integration

Documentation

An autogenerated documentation is available thanks to SwaggerUI at http://localhost:3000/docs.

• Generate the documentation.

  npm run swagger

CI/CD

A CI/CD process is set up thanks to GitLab CI/CD. Learn more about GitLab CI/CD via this page.

This one is described in the .gitlab-ci.yml. ⚠️ Right privileges must be granted to Docker on the session on which the CI is executed.

Linting

• Lint the application.

  npm run lint

Formatting

• Formatting the application.

  npm run format

🪛 Technical details

Technologies

• JavaScript
• Docker

Libraries

Project configuration

• expressjs is a backend NodeJS framework.
• body-parser is used for parsing REST API request body.
• cors is used for managing CORS.

Tests

• Jest is used for unit testing.
• SuperTest is used for integration testing.

Format

• eslint is used for linting the code.
• prettier is used for formatting the code.

Documentation

• swagger-autogen is used for SWAGGER documentation.
• swagger-ui-express is used UI SWAGGER documentation.

Tools

• npm is the package manager used.
• GitLab CI/CD is the CI/CD continuous tool used.
• Docker Desktop is the containerization technology used.
• Postman is the tool for testing manually the API.

🤝 Contributing

If you want to contribute to the project, please consider the following instructions:

• Any helping method or class must be named clearly (no abbreviations), especially integrating the type of detection, technology, and type of code fragment.
• More generally, any contribution must follow the conventions and keep the shape of previous contributions.
• Any contribution must be tested (unit and integration tests).
• All the tests and the CI/CD pipeline must pass before definitively integrating the contribution.
• Any contribution must be documented, especially by updating the README.md and the INSTALL.md file.
• Any contribution must be approved via the pull request mechanism.
• More generally, any contribution must follow the conventions and keep the shape of previous contributions.