Jdd 2016 DROP DATABASE

Jarosaw Ratajski

CSS Versicherung

DROP DATABASE

Jarek Ratajski

Developer, wizard, anarchitectLives in Luzern Works in CSS InsuranceC64, 6502, 68000, 8086, C++, Java, JEE, Scala, Akka, JS, [email protected] @jarek000000

Multiversum

MULTIVERSUM

Real life case

Galakpizza

10 000 Planets

3 Variants

3 Sizes

HAWAIIMARGHERITTAVEGETARIAN

MEDIUMLARGEXL

Domain summary

MARGHERITTA MEDIUM

HAWAII LARGE

HAWAII LARGE

VEGERARIAN LARGE

HAWAII XL

MARGHERITTA XL

HAWAII LARGE

HAWAII LARGE

HAWAII LARGE

HAWAII LARGE

3

3

1

1

Delivery model

10 000 Pizzas / sI want to look at number of queued orders 10000 times / sMy wife also!

Requirements

Alien development(in 2 parallel universes)

Normal universe

Other universe

++

In Normal universehttps://github.com/airomem/galakpizza/tree/master/galakpizza_n

Normal domain

public class Order implements Serializable {
private static final long serialVersionUID = 1L;
public final long id;
public final String planet;
public final Variant variant;
public final Size size;

public Order(long id, String planet, Variant variant, Size size) {
this.id = id;
this.planet = planet;
this.size = size;
this.variant = variant;
}
}

Normal domain 2

public enum Size {
MEDIUM,
LARGE,
XL
}public enum Variant {
HAWAII,
MARGHERITA,
VEGETARIAN
}

Normal Service

public interface GalakPizzaOrders {
long placeOrder(String planet, Variant variant, Size size);
}

Normal Service

}

public interface GalakPizzaDelivery {
List takeOrdersFromBestPlanet();

long countStandingOrders();
}

Normal Service

}


}public interface GalakPizzaService extends GalakPizzaDelivery, GalakPizzaOrders{
}

Normal service implementation...

public class GalakPizzaCore implements GalakPizzaService, Serializable, Storable {

public long placeOrder(String planet, Variant variant, Size size) { ...
}

public List takeOrdersFromBestPlanet() {
...
}

public long countStandingOrders() {
...
}

}


public class GalakPizzaCore implements GalakPizzaService, Serializable, Storable { private final Map orders = new HashMap();

}

...
}

...
}

}


public class GalakPizzaCore implements GalakPizzaService, Serializable, Storable { private final Map orders = new HashMap();

}

...
}

...
}

}

PlanetOrders - helper

class PlanetOrders implements Serializable {
final String name;

private List orders = new ArrayList();

public PlanetOrders(String name) {
this.name = name;
}

...
}


final String name;


this.name = name;
}

void assignOrder(final Order order) {
this.orders.add(order);
}

...
}


final String name;


this.name = name;
}

void assignOrder(final Order order) {
this.orders.add(order);
}

public boolean isEmpty() {
return this.orders.isEmpty();
}...
}

Normal place order

private final Map orders = new HashMap();

private long orderSequence = 1;

public long placeOrder(String planet, Variant variant, Size size) {
final long id = orderSequence++;
final Order order = new Order(id, planet, variant, size);
assignOrderToPlanet(order);

return id;
}

Normal place order




return id;
}

private void assignOrderToPlanet(Order order) {
final PlanetOrders po = orders.computeIfAbsent(order.planet,
planetName -> new PlanetOrders(planetName));
po.assignOrder(order);
...
}

Normal place order



private AtomicLong ordersTotal = new AtomicLong(0);

ordersTotal.incrementAndGet();
return id;
}

private void assignOrderToPlanet(Order order) {
...
}

Normal place order revisited

public class GalakPizzaCore { ...private final Map orders = new HashMap();private PriorityQueue bestPlanets = new PriorityQueue(256);private long orderSequence = 1;private AtomicLong ordersTotal = new AtomicLong(0);public long placeOrder(String planet, Variant variant, Size size) {
return id;
}private void assignOrderToPlanet(Order order) {
this.bestPlanets.offer(po);
}


return id;
this.bestPlanets.offer(po); BAD BAD BAD BAD
}


return id;
this.bestPlanets.offer(new PlanetOrders.Wrapper(po));
}

Wrapper... yeah

public static class Wrapper implements Comparable,Serializable{
final PlanetOrders porders;

final int size; //just keep it final
public Wrapper(PlanetOrders porders) {
this.porders = porders;
this.size = porders.orders.size();
}

@Override
public int compareTo(final Wrapper other) {
return other.size - this.size;
}
}

PlanetOrders take orders

final String name;


List takeOrders() {
final List result = this.orders;
this.orders = new ArrayList();
return result;
}

}}

PlanetOrders take orders

final String name;


List takeOrders() {
final List result = this.orders;
this.orders = new ArrayList(); //yes we MUTATE!
return result;
}

}}

CYPS 2(2) ZYPS 1(1) CYPS 2(2) ZYPS (1) ZYPS 2(2)+ Order 1 Hawaii XL to ZYPS

CYPS 2(2) ZYPS 1(0)Take Orders

ZYPS 1(0)Take Orders

IsEmpty => ignore

Normal take orders

private PriorityQueue bestPlanets = new PriorityQueue(256);
private AtomicLong ordersTotal = new AtomicLong(0); public List takeOrdersFromBestPlanet() {
final Optional planetOpt = takeBestPlanet();
.... }
private Optional takeBestPlanet() {
PlanetOrders.Wrapper planet = this.bestPlanets.poll();
while (planet != null && planet.porders.isEmpty()) {
planet = this.bestPlanets.poll();
}
return Optional.ofNullable(planet).map(p->p.porders);
}

Normal take orders

if (planetOpt.isPresent()) {
final PlanetOrders planet = planetOpt.get();
List orders = planet.takeOrders();

return orders;
}
return Collections.EMPTY_LIST;
}
}
}

Normal take orders

if (planetOpt.isPresent()) {
final PlanetOrders planet = planetOpt.get();
List orders = planet.takeOrders();
ordersTotal.addAndGet(-orders.size());
return orders;
}
}
}
}

Normal count orders

private AtomicLong ordersTotal = new AtomicLong(0);public long countStandingOrders() {
return this.ordersTotal.get();
}

We have service - but do we deal with?

Concurrency

Persistence

Normal - Alien service wrapper

public class GalakPizza implements GalakPizzaService {
final PersistenceController controller;

public GalakPizza() {
controller = PrevaylerBuilder.newBuilder()
.withinUserFolder("pizza")
.useSupplier( () -> new GalakPizzaCore())
.build();
} public long placeOrder(final String planet, final Variant variant, final Size size) {
return controller.executeAndQuery( core -> core.placeOrder(planet,variant,size));
}
return controller.executeAndQuery( core ->core.takeOrdersFromBestPlanet());
}

return controller.query( c->c.countStandingOrders());
}
...
}

Done!

In alternative universehttps://github.com/airomem/galakpizza/tree/master/galakpizza_h

Other Domain

@Entity
@Table(name="T_ORDER")
@NamedQueries( {
@NamedQuery(name = "select best planet", query = "SELECT o.planet, count(o) FROM Order o " +
" GROUP BY o.planet ORDER BY count(o) desc"),
@NamedQuery(name = "select orders", query = "SELECT o FROM Order o WHERE o.planet = :planet"),
@NamedQuery(name = "count orders", query = "SELECT count(o) FROM Order o ")

})
public class Order {

@Id
@GeneratedValue(generator="increment")
@GenericGenerator(name="increment", strategy = "increment")
final long id;

public final String planet;
public final Variant variant;
public final Size size;

protected Order() {
this("this is strange...", null, null);
}

public Order(String planet, Variant variant, Size size) {
this.size = size;
id = 0;
}

public long getId() {
return id;
}

}

Other domain (same as above)

public enum Size {
MEDIUM,
LARGE,
XL
}public enum Variant {
HAWAII,
MARGHERITA,
VEGETARIAN
}

It is in TABLE so it exists

Other service (same)

}


}public interface GalakPizzaService extends GalakPizzaDelivery, GalakPizzaOrders{
}

Select best blanet...

SELECT o.planet, count(o) FROM Order o

GROUP BY o.planet ORDER BY count(o) desc

Other Planet (but almost same story)

@Entity
@Table(name="T_PLANET",
indexes = {@Index(name="cnt_idx", columnList = "count", unique = false)} )
@NamedQueries( {
@NamedQuery(name = "select best planet from table", query = "SELECT p FROM Planet p " +
" ORDER BY p.count desc")})
public class Planet {
@Id
public final String name;

private int count;

protected Planet() {
name = "default";
}

public Planet(String name) {
this.name = name;
}

public int getCount() {
return this.count;
}

public void increment() {
this.count++;
}

public void clear() {
this.count = 0;
}
}

Other service implementation


final SessionFactory sessionFactory;

public GalakPizza(SessionFactory sessionFactory) {
this.sessionFactory = sessionFactory;
}

...
}

...
}

public long countStandingOrders() { ...
}
}

Other - Place order

return this.runInSession(session -> {
final Order orderEntity = new Order(planet, variant, size);
final Long key = (Long) session.save(orderEntity);
incrementPlanetCounter(planet, session);
return key;
});
}
private void incrementPlanetCounter(String planet, Session session) {
Planet planetEntity = session.get(Planet.class, planet);
if (planetEntity == null) {
planetEntity = new Planet(planet);
session.save(planetEntity);
}
planetEntity.increment();
}private T runOnSession(Function dbCommand) { ...}

Other - runInSession

private T runInSession(Function dbCommand) {
final Session session = sessionFactory.openSession();
session.beginTransaction();
try {
final T result = dbCommand.apply(session);
session.getTransaction().commit();
session.close();
return result;
} catch (ConstraintViolationException cve) {
session.getTransaction().rollback();
session.close();
return runInSession(dbCommand);
} catch (Throwable t) {
t.printStackTrace();
throw new RuntimeException(t);
}
}

So easy

Other - Take orders

final Query bestPlanetQuery = session.getNamedQuery("SELECT p FROM Planet p " +
" ORDER BY p.count desc");
bestPlanetQuery.setMaxResults(1);
final Iterator bestPlanetsIterator = bestPlanetQuery.iterate();
if (bestPlanetsIterator.hasNext()) {

final Planet bestOne = bestPlanetsIterator.next();
final Query ordersQuery = session.getNamedQuery("SELECT o FROM Order o WHERE o.planet = :planet");
ordersQuery.setParameter("planet", bestOne.name);
final List orders = ordersQuery.list();

orders.forEach(o -> session.delete(o));
bestOne.clear();
return orders;
}
});
}

Other count orders

@Override
final Query ordersCount = session.getNamedQuery("count orders");
final Long cnt = (Long) ordersCount.iterate().next();
return cnt;
});
}

Other - others

Hibernate config, init

MySql config

After one day of coding...

In a real galaxy

That we can simulate as

Customers - 4 Threads doing in loop placeOrder (random planet name, variant and size) no pauses

PizzBoy - 1 Thread doing in loop takeBestOrder no pauses

Owners 2 Threads doing in loop countStandingOrders

Galaxy simulation

public static void main(String... args) {
runSimulation(5000);
System.gc();
sleep();
Result result = runSimulation(SECONDS_TOTAL*1000);
System.out.println("Orders processed:" + result.processed);
System.out.println("p/s:" + (double)result.processed/(double) SECONDS_TOTAL);
System.out.println("Orders seen:" + result.seen);
System.out.println("o/s:" + (double)result.seen/(double) SECONDS_TOTAL);
}

?

Normal universe

Other universe

~40 000 p/s

~600 p/s

Processed orders

Normal universe

Other universe

~ 12 823 500 p/s

~6362 p/s

Peeping speed

Normal universe

Other universe

Normal universe

Other universe

GODS of SQL HELP! ME

Normal universe

Other universe

~60 000 p/s

~1000 p/s

After few days of operations...

Normal universe

Other universe

Cool, that it runs onlaptop

Reality strickes back

I do not trust PizzaBoyI want to know where exactly were pizzas delivered

Normal universe

Other universe

Cool, we will not delete orders any more, only mark them as deleted!Cool, we will add a list that will be processed during takeOrders,or maybe we will store data in DB Table (as archive)

Reality strickes back

I do not trust PizzaBoyI want to know where were pizzas deliveredYesterday!!!

Normal universe

Other universe

Crap...We will restart system and commands will be reprocessed

Commands once again

final PersistenceController controller;

return controller.executeAndQuery( core ->core.takeOrdersFromBestPlanet());
}

return controller.query( c->c.countStandingOrders());
}

public long placeOrder(final String planet, final Variant variant, final Size size) {
return controller.executeAndQuery( core -> core.placeOrder(planet,variant,size));
}
}

Yet another real world case

System evolves

I want to let people compose their pizza/Like: more cheese on one half, and no meat on the next half, And onion on the third half.

Normal universe

Other universe

We will map it with only 20 tablesOr do we add TEXT field?public Order(long id, String planet, Supplier pizzaDescription, Size size) {

public Order(long id, String planet, Variant variant, Size size) {

We will use factory

Difference

+ Fast

+ Code in Java

+ Strongly typed

+ Flexible

+ Testable

- Sophisticated java

- Terribly slow

- Code in Strings and Annotations- Stringly typed

- Limited domain

- Testing expensive

+ Easy java

Why different ?

Long time ago

Sweet spot known as 90's

Computers had mostly one CPU Core

Memory (RAM) was expensive but more as 100MB available in single unit

Disks were slow but huge (like 2Gb-20Gb)

Time for JEE

Around galactic year 2003

Normal universe

RAM is so cheap and huge

Computers have multiple cores

Alien developers start to use full power of hardware with Java

No need to store data in databases anymore

Because RAM is so cheap and huge

Other universe

RAM is cheap and huge

Computers have multiple cores

But alien developers have never realised that

Other universe is a good place

Database vendors earn money and are happy

Application Servers vendors earn money and are happy

Power plants are happy

Hardware vendors are happy

Only business complains but they still pay

Developers have a lot more work to do they are happy..

Gfx By pressfoto / Freepik

DB / ORM vs Prevalence Problems

ProblemPrevalent (Normal)ORMs (Other)

Detached objectsYesYes

Transaction isolationYes in JAVA(concurrency)Yes in DB

TestingTrivial Awful

CacheIt is all cacheLots of configs...

MagicContainedEverywhere

MappingAll serializable goesLimited

If you want to jump to normal universe...

Simple start

Prevayleror

Airomem (Prevayler wrapper for Java 8 )or

Nothing :-)

Airomem quick - maven

pl.setblack
airomem-core
1.0.5

Airomem - init

PersistenceController controller =controller = PrevaylerBuilder
.newBuilder()
.withinUserFolder("pizza")
.useSupplier( () -> new GalakPizzaCore())
.build();

GalakPizzaCore is main/plain object - Root aggregate!Must be Serializable! (how bad)

Airomem commands

controller.executeAndQuery( core->core.placeOrder(planet, variant, size )); ----------------------controller.executeAndQuery( new PlaceOrderCommand(planet,variant,size));

private static final class PlaceOrderCommand implements Command {
private final String planet;
private final Variant variant;
private final Size size;
public PlaceOrderCommand(String planet, Variant variant, Size size){
this.size = size;
}

public Long execute(GalakPizzaCore system) {
return system.placeOrder(planet,variant, size);
}
}

Commands are never executed concurrently

Query

controller.query( c->c.countStandingOrders());

Queries are executed concurrently - also with one command!

Airomem snapshots, events

jarek@ubuntu:~$ cd prevayler/jarek@ubuntu:~/prevayler$ lspizzajarek@ubuntu:~/prevayler$ cd pizza/jarek@ubuntu:~/prevayler/pizza$ ls -altotal 188drwxrwxr-x 2 jarek jarek 4096 May 6 14:46 .drwxrwxr-x 3 jarek jarek 4096 Apr 26 13:40 ..-rw-rw-r-- 1 jarek jarek 675 Apr 26 14:17 0000000000000000001.journal-rw-rw-r-- 1 jarek jarek 3375 Apr 26 14:18 0000000000000000002.journal-rw-rw-r-- 1 jarek jarek 3369 May 5 10:51 0000000000000000007.journal-rw-rw-r-- 1 jarek jarek 683 May 5 13:48 0000000000000000012.journal-rw-rw-r-- 1 jarek jarek 2048 May 5 14:12 0000000000000000012.snapshot-rw-rw-r-- 1 jarek jarek 497 May 5 14:12 0000000000000000013.journal-rw-rw-r-- 1 jarek jarek 1849 May 5 14:15 0000000000000000013.snapshot-rw-rw-r-- 1 jarek jarek 497 May 5 14:15 0000000000000000014.journal-rw-rw-r-- 1 jarek jarek 1685 May 5 14:17 0000000000000000014.snapshot-rw-rw-r-- 1 jarek jarek 8086 May 5 14:19 0000000000000000015.journal-rw-rw-r-- 1 jarek jarek 1084 May 6 13:25 0000000000000000028.snapshot-rw-rw-r-- 1 jarek jarek 57977 May 6 14:05 0000000000000000029.journal-rw-rw-r-- 1 jarek jarek 1228 May 6 14:46 0000000000000000132.snapshot-rw-rw-r-- 1 jarek jarek 59021 May 8 04:28 0000000000000000133.journal

Problems?

RAM?

Data migration

Not big deal in fact

Java Serialization (sth to learn)

readResolve, writeReplace

XML export option

Transaction

This is always transactional!!!

public Long execute(GalakPizzaCore system) {
return system.placeOrder(planet,variant, size);
}

Concurrency

Learn java.util.concurrent, synchronized or die!

Replication/ failover

Not in standard... butTrivial to implement - send events to other machines, share folder

Indices

Indices in memory do existsCQEngineQuery query1 = or(endsWith(Car.NAME, "vic"), lessThan(Car.CAR_ID, 2));for (Car car : cars.retrieve(query1)) { System.out.println(car);}

Airomem summary

CQRS + ES simplified - one command one eventWith RAM projection

More enterprisy alternatives

Lagom!

Akka persistence

CQRS generally

Advice

Remember to add Thread.sleep(1000) often

Storing in Prevayler vs DB

model events vs model domain

store facts vs store state

Shared state is a cache vs shared state is a problem

Domain with little magic vs domain polluted

Clean architecture vs Maybe DDD can help...

End

Try on something small first (not bigger then one planet)

Drop DB from core of your system

Drop your application servers

Drop annotations

Drop magic

Learn Java and algorithms

Feel the power of RAM and CPU alltogether!

Jdd 2016 DROP DATABASE

Software

Transcript of Jdd 2016 DROP DATABASE