Modularized Persistence - B Zsoldos
-
Upload
mfrancis -
Category
Technology
-
view
155 -
download
1
description
Transcript of Modularized Persistence - B Zsoldos
Balázs Zsoldos
Modularized Persistence
History of NoSQL by Mark Madsen, Picture published by Edd Dumbill
Blueprint+
JPA+
JSF
DeclarativeServices
+ModularizedPersistence
+JSF
ECM+
ModularizedPersistence
+JSF
ECM+
ModularizedPersistence
+Modularized
Web
DeclarativeServices
+JPA
+JSF
Stefan Seifert – Apache Sling & Friends Tech Meetup, Berlin 2012
JDBC Driver(DataSourceFactory)
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
???
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
Liquibase DataSource Component(DataSource)
<databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" logicalFilePath="org.everit.osgi.resource.ri.schema" xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd">
<changeSet id="1.0.0" author="everit">
<createTable tableName="res_resource"> <column name="resource_id" type="bigint" autoIncrement="true"> <constraints primaryKeyName="pk_res_resource" primaryKey="true" nullable="false" /> </column> </createTable> </changeSet>
</databaseChangeLog>
Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml";
Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml"
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
Liquibase DataSource Component(DataSource)
Business Component Business Component Business ComponentQuerydsl Querydsl Querydsl
Examples from http://querydsl.com
Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata
<lqmg xmlns="http://everit.org/lqmg" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" defaultPackage="org.everit.osgi.resource.ri.schema.qdsl"> <namingRules> <classNameRule> <entity>res_resource</entity> <class>Resource</class> <propertyMappings> <primaryKey> <name>pk_res_resource</name> <property>resourcePk</property> </primaryKey> </propertyMappings> </classNameRule> </namingRules></lqmg>
Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata
Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml";
Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml"
Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml"; lqmg.config.resource="/META-INF/liquibase/resource.ri.lqmg.xml"
Provide-Capability: liquibase.schema;name="org.everit.osgi.resource.ri"; resource="/META-INF/liquibase/resource.ri.liquibase.xml";lqmg.config.re source="/META-INF/liquibase/resource.ri.lqmg.xml"
Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata
<databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" logicalFilePath="org.everit.osgi.authorization.ri.schema" xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">
<include file="eosgi:org.everit.osgi.resource.ri" /> <include file="eosgi:org.everit.osgi.props.ri" />
<changeSet id="1.0.0" author="everit">
<createTable tableName="authr_permission"> <column name="authorized_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_p_a" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> <column name="target_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_p_t" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> <column name="action_" type="varchar(255)"> <constraints nullable="false" /> </column> </createTable> <addPrimaryKey constraintName="pk_authr_perm" tableName="authr_permission" columnNames="authorized_resource_id,target_resource_id,action_" />
<createTable tableName="authr_permission_inheritance"> <column name="parent_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_pi_p" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> <column name="child_resource_id" type="bigint"> <constraints nullable="false" foreignKeyName="fk_res_r_authr_pi_c" referencedTableName="res_resource" referencedColumnNames="resource_id" /> </column> </createTable> <addPrimaryKey constraintName="pk_authr_perm_inheritance" tableName="authr_permission_inheritance" columnNames="parent_resource_id,child_resource_id" /> </changeSet>
</databaseChangeLog>
<include file="eosgi:org.everit.osgi.resource.ri" /> <include file="eosgi:org.everit.osgi.props.ri" />
Resource
Authorization
Provide-Capability: liquibase.schema; name="org.everit.osgi.resource.ri"; ...
Require-Capability: liquibase.schema; filter:=(name=org.everit.osgi.resource.ri)
<databaseChangeLog ...>
<include file="eosgi:org.everit.osgi.resource.ri" />
...
</databaseChangeLog>
Write Liquibase changelog
Write LQMG file
Add LQMG to the Capability
Add inclusions
Generate Querydsl Metadata
Start embeddedOSGi container
Deploy modules
Start embeddedH2 database
Initialize databaseschema
Generate QuerydslMetadata
LQMG
Evil magic here!!!
If Capability is not found after deployments, unsatisfied bundles are enhanced in the way that unsatisfied requirements are marked to be optional.
Use-cases???
Monoholitic
Authorization
Authentication
User
Documents
Articles
SiteMap
Blog
Portal(v5.1.2)
Modularized
Portal
Authorization (v1.1.0)
Authentication (v2.0.0)
User (v1.3.2)
Documents (v1.0.1)
SiteMap (v1.0.3)
Articles (v3.0.14)
Blog (v2.11.0)
User
User Address
User Address Country
User Address Country
Company
REAL PROJECT!!!
Authorization
Permission
● authorized_resource_id● action● target_resource_id
Resource
● resource_id
Permission Inheritance
● parent_resource_id● child_resource_id
SELECT ... FROM document d JOIN document.attachment a WHERE … LIMIT 10 OFFSET 1000;
EXISTS(SELECT 1 FROM permission p WHERE p.authorized_resource_id IN (?, …, ?) AND p.target_resource_id = d.resource_id AND action = ?)
SELECT ... FROM document d JOIN document.attachment a WHERE EXISTS(SELECT 1 FROM permission p WHERE p.authorized_resource_id IN (?, …, ?) AND p.target_resource_id = d.resource_id AND action = ?) AND … LIMIT 10 OFFSET 1000;
@Override public BooleanExpression authorizationPredicate(final long authorizedResourceId, final Expression<Long> targetResourceId, final String... actions) { if (authorizedResourceId == systemResourceId) { return BooleanTemplate.TRUE; }
Objects.requireNonNull(targetResourceId, "Parameter targetResourceId must not be null"); validateActionsParameter(actions);
long[] authorizationScope = getAuthorizationScope(authorizedResourceId);
SQLSubQuery subQuery = new SQLSubQuery();
QPermission permission = QPermission.permission;
BooleanExpression authorizedResourceIdPredicate; if (authorizationScope.length == 1) { authorizedResourceIdPredicate = permission.authorizedResourceId.eq(authorizationScope[0]); } else { // More than one as the scope contains at least one value (other branch) Long[] authorizationScopeLongArray = new Long[authorizationScope.length]; for (int i = 0, n = authorizationScope.length; i < n; i++) { if (authorizationScope[i] == systemResourceId) { return BooleanTemplate.TRUE; } authorizationScopeLongArray[i] = authorizationScope[i]; }
authorizedResourceIdPredicate = permission.authorizedResourceId.in(authorizationScopeLongArray); }
BooleanExpression actionPredicate = null;
if (actions.length == 1) { actionPredicate = permission.action.eq(actions[0]); } else { actionPredicate = permission.action.in(actions); }
return subQuery.from(permission) .where(permission.targetResourceId.eq(targetResourceId).and( actionPredicate.and(authorizedResourceIdPredicate))) .exists(); }
SQLQuery query = new SQLQuery(connection, configuration);QDocument document = new QDocument("d");
BooleanExpression permissionCheckPredicate = authorizationQdslUtil .authorizationPredicate(userId, document.resourceId, actions);
List<Long> list = query.from(document).where(permissionCheckPredicate) .list(document.documentId);
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
Liquibase DataSource Component(DataSource)
Business Component Business Component Business Component
QuerydslSupport
QuerydslConfiguration
QuerydslSQLTemplates
Querydsl Querydsl Querydsl
public List<Long> listDocumentIds() {
return qdsl.execute((connection, configuration) -> {
SQLQuery query = new SQLQuery(connection, configuration); QDocument document = new QDocument("d");
BooleanExpression permissionCheckPredicate = authorizationQdslUtil .authorizationPredicate(a1, document.resourceId, actions);
List<Long> list = query.from(document).where(permissionCheckPredicate) .list(document.documentId); }}
@Reference(name = "querydslSupport", bind = "setQdsl")private QuerydslSupport qdsl;
public void setQdsl(QuerydslSupport qdsl) { this.qdsl = qdsl;}
JDBC Driver(DataSourceFactory)
DSF Component(XADataSource)
DBCP Component(DataSource)
Liquibase DataSource Component(DataSource)
Business Component Business Component Business Component
QuerydslSupport
QuerydslConfiguration
QuerydslSQLTemplates
Querydsl Querydsl Querydsl
Transaction Helper
public long saveUser(String firstName, String lastName) { Objects.requireNonNull(firstName); Objects.requireNonNull(lastName);
return transactionHelper.required(() -> { // Logic that should be implemented return 0l; }); }
Caching
● Available Cache modules: cache-api, cache-infinispan, cache-noop
● Caching should be done in the persistent module by the programmer, who knows the business logic
● Caching and table updates should be done within the same component
● In case bulk update is done in another module, cache must be cleared
OpenSource modules
● Resource● Authorization● Authentication● Property Manager● Blobstore (Before review)● Audit (Before review)
Roadmap
HTML Output
LQMG / Maven
H2
Liquibase schema(Bundle-Capability)
HTML
LQMG / Maven
Database
Liquibase schema(Bundle-Capability)
Generate Schema
Manual Schema update
Webconsole plugin
DataSource(OSGi service)
Liquibase schema(Bundle-Capability)
Generate SchemaCustom User / Password
Manual Schema generation
Dump SQL file
Webconsole plugin
DataSource(OSGi service)
Liquibase schema(Bundle-Capability)
SQL
LQMG extension in changelog.xml
<databaseChangeLog objectQuotingStrategy="QUOTE_ALL_OBJECTS" logicalFilePath="org.everit.osgi.resource.ri.schema" xmlns="http://www.liquibase.org/xml/ns/dbchangelog" xmlns:lqmg="http://everit.org/xml/ns/lqmg" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.0.xsd">
<changeSet id="1.0.0" author="everit">
<createTable tableName="res_resource" lqmg:class="QResource"> <column name="resource_id" type="bigint" autoIncrement="true" lqmg:property="resourceId"> <constraints primaryKeyName="pk_res_resource" primaryKey="true" nullable="false" /> </column> </createTable> </changeSet>
</databaseChangeLog>
Full Text Search
● Analize the different solutions– H2 → Lucene
– MySQL → Sphinx
– PostgreSQL → TSearch2
– SQL Server →???– Oracle → Oracle Text
● Create a common API for Querydsl based query extension
● Create a module for each database engine
Querydsl: http://www.querydsl.com/
Liquibase: http://www.liquibase.org/
Liquibase OSGi Bundle: https://github.com/everit-org/osgi-liquibase-bundle
LQMG: https://github.com/everit-org/osgi-lqmg
LQMG Maven Plugin: http://www.everit.org/lqmg-maven-plugin/
Liquibase DataSource: https://github.com/everit-org/osgi-liquibase-datasource
Querydsl SQLTemplates Component: https://github.com/everit-org/osgi-querydsl-templates
Querydsl Configuration Component: https://github.com/everit-org/osgi-querydsl-configuration
Querydsl Support Component: https://github.com/everit-org/osgi-querydsl-support
Cookbook chapter: http://cookbook.everit.org/persistence/index.html
Download from Maven central
Resource: https://github.com/everit-org/resource-ri
Authorization: https://github.com/everit-org/authorization-ri
Authentication: https://github.com/everit-org/authentication-simple
Property Manager: https://github.com/everit-org/property-manager-ri
Twitter: @EveritOrg