Intro
Developing systems that has a lot of content storing and retrieval tasks, can involve a lot of repetitive and boring activities, let’s assume that you are developing an E-commerce platform for a customer, and start asking you to develop a feature for adding fields to products, or adding a new photo sliders to the home page, if you are using your own CRUD system, this will require you more time, and you will be lest focused on the business logic modelling and implementation.
Let’s assume that for some reason, you are required to change certain UI elements in the admin control panel, or you have changed the way you store the photos in the servers, then you will have to do it in all of the modules you have already developed.
We aim here to develop a way that automates the generation of content management systems components in your system, so the developers can get more time to focus on the business logic modeling activities and reduce the time for modifications.
Why not using CMS?
CMS provides the same purpose here, but considering performance; these systems uses the same tables and fields to represent entities, we are assuming that we have our own database and separate table for each entity, and our own system architecture, infrastructure, and design patterns.
The idea
Assuming we have the system structure and framework that we are developing the system with (in the example the target system is developed in Laravel).
We will need:
1. Modeling our entities (tables, columns, and their constraints) and their actions and relations (We will use JSON files in the example).
2. Creating templates for our target-generated files (Laravel components: Models, Controllers, Requests).
3. Parsing the entity descriptions and generating the modules based on the templates.
The example:
A simple blog package with an Author entity and Post entity, the Author has many Posts.
Modeling the entities:
We can represent the entity, its fields, relations, and actions with simple JSON files.
author.json
{
"entity": "author",
"id": true,
"id_field": "id",
"id_generation": "SEQUENCE",
"date_modified": true,
"date_created": true,
"fields": [
{
"name": "name",
"type": "TEXT",
"input_type": "TEXT",
"nullable": false,
"pattern": null,
"default_value": null,
"fillable": true,
"unique": false,
"hidden": false,
"index": false,
"meta": {},
"translatable": false
}
],
"belongs_to": [],
"has_many": [
{
"entity": "post",
"name": "posts",
"foreign_key": "author_id",
"is_many_to_many": false,
"many_to_many_table": null,
"other_foreign_key": null,
"attach_on_create": false,
"create_on_create": false,
"sync": false,
"detachable": false
}
],
"requests": [
{
"type": "GET",
"required_permissions": [],
"required_auth": false,
"meta": {},
"with": []
},
{
"type": "GET_ONE",
"required_permissions": [],
"required_auth": false,
"meta": {},
"with": []
},
{
"type": "CREATE",
"required_permissions": [],
"required_auth": false,
"meta": {},
"with": []
},
{
"type": "UPDATE",
"required_permissions": [],
"required_auth": false,
"meta": {},
"with": []
},
{
"type": "DELETE",
"required_permissions": [],
"required_auth": false,
"meta": {},
"with": []
}
]
}
posts.json
{
"entity": "post",
"id": true,
"id_field": "id",
"id_generation": "SEQUENCE",
"date_modified": true,
"date_created": true,
"fields": [
{
"name": "title",
"type": "MEDIUM_TEXT",
"input_type": "TEXT",
"nullable": false,
"pattern": null,
"default_value": null,
"fillable": true,
"unique": false,
"hidden": false,
"index": false,
"meta": {},
"translatable": false
}, {
"name": "body",
"type": "LONG_TEXT",
"input_type": "WYSIWYG",
"nullable": false,
"pattern": null,
"default_value": null,
"fillable": true,
"unique": false,
"hidden": false,
"index": false,
"meta": {},
"translatable": false
}
],
"belongs_to": [
{
"entity": "author",
"name": "author",
"foreign_key": "author_id",
"foreign_key_nullable": false,
"attach_on_create": true,
"on_delete": "CASCADE"
}
],
"has_many": [],
"requests": [
{
"type": "GET",
"required_permissions": [],
"required_auth": false,
"meta": {
"paginated": true,
"per_page": 10
},
"with": ["author"]
},
{
"type": "GET_ONE",
"required_permissions": [],
"required_auth": false,
"meta": {},
"with": ["author"]
},
{
"type": "CREATE",
"required_permissions": [],
"required_auth": false,
"meta": {},
"with": ["author"]
},
{
"type": "UPDATE",
"required_permissions": [],
"required_auth": false,
"meta": {},
"with": ["author"]
},
{
"type": "DELETE",
"required_permissions": [],
"required_auth": false,
"meta": {},
"with": ["author"]
}
]
}
Parsing the entities
We can use the following jackson-databind package, we need to create a model for our JSON schema, read the JSON file and map it to an Entity object.
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.14.0</version>
</dependency>
The following represent the Entity schema.
package model.entities;
import freemarker.template.utility.StringUtil;
import lombok.*;
import model.entities.fields.Field;
import model.entities.relations.BelongsTo;
import model.entities.relations.HasMany;
import model.requests.Request;
import org.atteo.evo.inflector.English;
import java.util.List;
@Setter @Getter @Builder @NoArgsConstructor @AllArgsConstructor
public class Entity {
private String entity;
private Boolean id;
private String id_field;
private IdGeneration id_generation;
private Boolean date_modified;
private Boolean date_created;
private List<Field> fields;
private List<BelongsTo> belongs_to;
private List<HasMany> has_many;
private List<Request> requests;
public String getEntityPlural() {
return English.plural(entity);
}
public String getEntityClass() {
return StringUtil.capitalize(entity);
}
}
The Parser
EntityParser is used to read the file from the resources path and parse it into the Entity object.
package parsers;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import model.entities.Entity;
import java.io.IOException;
import java.nio.file.Path;
import java.nio.file.Paths;
public class EntityParser {
public Entity parseJson(String json) throws JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper();
return objectMapper.readValue(json, Entity.class);
}
public Entity parseFileFor(String module, String entity) throws IOException {
return parseFileForPath(Paths.get("src","main","resources", module, entity.concat(".json")));
}
private Entity parseFileForPath(Path path) throws IOException {
ObjectMapper objectMapper = new ObjectMapper();
return objectMapper.readValue(path.toFile(), Entity.class);
}
}
The Templates
Our example here is generating the Laravel Blog module which has the DB migrations, the models, controllers, etc.…
We are using FreeMarker Java Template Engine for writing the templates for the components in .ftlh files, checking the guides and docs can help us understand how it works.
Adding the dependency:
<dependency>
<groupId>org.freemarker</groupId>
<artifactId>freemarker</artifactId>
<version>2.3.31</version>
</dependency>
migration.ftlh
<#noautoesc>${"<?php"}</#noautoesc>
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
public function up()
{
Schema::create('${entity.getEntityPlural()}', function (Blueprint $table) {
<#if entity.id>
$table->id();
</#if>
<#list entity.fields as field>
$table->
<#if field.type == 'TEXT' >
string
</#if>
<#if field.type == 'MEDIUM_TEXT' >
mediumText
</#if>
<#if field.type == 'LONG_TEXT' >
longText
</#if>
('${field.name}');
</#list>
<#list entity.belongs_to as master>
$table->unsignedBigInteger('${master.foreign_key}');
$table->foreign('${master.foreign_key}')->references('id')->on('${master.getEntityPlural()}')
<#if master.on_delete == 'CASCADE'>->onDelete('cascade')</#if>
;
</#list>
<#if entity.date_modified || entity.date_created>$table->timestamps();</#if>
});
}
public function down()
{
Schema::dropIfExists('${entity.getEntityPlural()}');
}
};
model.ftlh
<#noautoesc>${"<?php"}</#noautoesc>
namespace App\${packageName}\Models;
use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;
class ${entity.getEntityClass()} extends Model
{
protected $fillable = [${fillable?map(f -> "'${f}'")?join(',')?no_esc}];
protected $hidden = [${entity.fields?filter(f -> f.hidden)?map(f -> "'${f.name}'")?join(',')?no_esc}];
protected $table = '${entity.getEntityPlural()}';
use HasFactory;
<#list entity.belongs_to as master>
public function ${master.name}() {
return $this->belongsTo(${master.getEntityClass()}::class, '${master.foreign_key}');
}
</#list>
<#list entity.has_many?filter(c -> !c.is_many_to_many) as child>
public function ${child.name}() {
return $this->hasMany(${child.getEntityClass()}::class, '${child.foreign_key}');
}
</#list>
}
controller.ftlh
<#noautoesc>${"<?php"}</#noautoesc>
namespace App\${packageName}\Controllers;
use App\${packageName}\Models\${entity.getEntityClass()};
use App\${packageName}\Requests\${entity.getEntityClass()}Request;
use App\Http\Controllers\Controller;
use Illuminate\Http\Request;
class ${entity.getEntityClass()}Controller extends Controller
{
<#list entity.requests as request>
<#if request.type == "GET">
public function index() {
$data = ${entity.getEntityClass()}::with([${request.with?map(f -> "'${f}'")?join(',')?no_esc}])
<#if request.meta.paginated == 'true'>->paginate(${request.meta.per_page})</#if>
;
return response()->json($data);
}
</#if>
<#if request.type == "GET_ONE">
public function get($id) {
$data = ${entity.getEntityClass()}::query()->with([${request.with?map(f -> "'${f}'")?join(',')?no_esc}])->findOrFail($id);
return response()->json($data);
}
</#if>
<#if request.type == "CREATE">
public function create(${entity.getEntityClass()}Request $request) {
$data = new ${entity.getEntityClass()}($request->all());
$data->save();
$data = ${entity.getEntityClass()}::query()->with([${request.with?map(f -> "'${f}'")?join(',')?no_esc}])->findOrFail($data->id);
return response()->json($data);
}
</#if>
<#if request.type == "UPDATE">
public function update($id, ${entity.getEntityClass()}Request $request) {
$data = ${entity.getEntityClass()}::query()->findOrFail($id);
$data->update($request->all());
$data = ${entity.getEntityClass()}::query()->with([${request.with?map(f -> "'${f}'")?join(',')?no_esc}])->findOrFail($data->id);
return response()->json($data);
}
</#if>
<#if request.type == "DELETE">
public function delete($id) {
$data = ${entity.getEntityClass()}::query()->with([${request.with?map(f -> "'${f}'")?join(',')?no_esc}])->findOrFail($id);
$data->delete();
return response()->json($data);
}
</#if>
</#list>
}
In the full code, you can check the rest of the templates for different components.
Generating the file using templates
The following purpose is the set up the model for generating the DB migration file.
package generator;
import freemarker.template.Template;
import freemarker.template.TemplateException;
import model.entities.Entity;
import java.io.IOException;
import java.io.StringWriter;
import java.io.Writer;
import java.util.HashMap;
import java.util.Map;
public class MigrationGenerator {
private static final String template = "migration.ftlh";
public String generate(Entity entity) throws IOException, TemplateException {
/* Create a data-model */
Map root = new HashMap();
root.put("entity", entity);
/* Get the template (uses cache internally) */
Template temp = TemplateConfig.getConfig().getTemplate(template);
/* Merge data-model with template */
Writer out = new StringWriter();
temp.process(root, out);
out.close();
return out.toString();
}
}
the result will be (after replacing the entity fields in the template):
<?php
namespace App\Blog\Models;
use Illuminate\Database\Eloquent\Factories\HasFactory;
use Illuminate\Database\Eloquent\Model;
class Post extends Model
{
protected $fillable = ['title', 'body', 'author_id'];
protected $hidden = [];
protected $table = 'posts';
use HasFactory;
public function author() {
return $this->belongsTo(Author::class, 'author_id');
}
}
And the controller:
<?php
namespace App\Blog\Controllers;
use App\Blog\Models\Post;
use App\Blog\Requests\PostRequest;
use App\Http\Controllers\Controller;
use Illuminate\Http\Request;
class PostController extends Controller
{
public function index() {
$data = Post::with(['author'])->paginate(10);
return response()->json($data);
}
public function get($id) {
$data = Post::query()->with(['author'])->findOrFail($id);
return response()->json($data);
}
public function create(PostRequest $request) {
$data = new Post($request->all());
$data->save();
$data = Post::query()->with(['author'])->findOrFail($data->id);
return response()->json($data);
}
public function update($id, PostRequest $request) {
$data = Post::query()->findOrFail($id);
$data->update($request->all());
$data = Post::query()->with(['author'])->findOrFail($data->id);
return response()->json($data);
}
public function delete($id) {
$data = Post::query()->with(['author'])->findOrFail($id);
$data->delete();
return response()->json($data);
}
}
Testing
We will write a fully working code file to a test file as an expected result, read the expected result file, then generate the code after parsing and using the template, comparing both to check the correctness of the template and generating logic.
package generator;
import model.entities.Entity;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import parsers.EntityParser;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import static org.junit.jupiter.api.Assertions.*;
class ModelGeneratorTest {
@Test
@DisplayName("Model generated correctly for posts")
void generateForPosts() {
ModelGenerator generator = new ModelGenerator();
assertAll(() -> {
String expected = getExpected("post.txt");
EntityParser parser = new EntityParser();
Entity e = parser.parseFileFor("blog", "posts");
String generated = generator.generate(e, "Blog").replaceAll("\\s", "");
assertEquals(expected, generated);
});
}
@Test
@DisplayName("Model generated correctly for authors")
void generateForAuthors() {
ModelGenerator generator = new ModelGenerator();
assertAll(() -> {
String expected = getExpected("author.txt");
EntityParser parser = new EntityParser();
Entity e = parser.parseFileFor("blog", "author");
String generated = generator.generate(e, "Blog").replaceAll("\\s", "");
assertEquals(expected, generated);
});
}
private String getExpected(String file) throws IOException {
return new String(
Files.readAllBytes(
Paths.get("src","test", "resources", "templates", "models", file)
))
.replaceAll("\\s", "");
}
}
Note
We can apply the same steps to generate the UI, not necessarily for the same Laravel Project, we can generate Angular Components!
Benefits
1- We have a general architecture for our CMS and we want to apply it in each module.
2- We can be using our in-house developed framework and infrastructure, not a general-purpose CMS.
3- Maintenance: an update to one component requires updating the template only and regeneration of all modules.
4- Reusability and easy migration: we can change the target framework only by changing the template, the model of the entities can be still the same.
5- One input several outputs: one description, multiple modules for multiple projects (e.g. backend modules and frontend modules.)
6- More time to focus on Business Logic and less time for CRUD modules.