Field Storage API

FieldableEntityStorageController

CCK

Artem Sylchuk, IntenetDevels

About me:

Drupal developer since 2010

Story 1Save 1 000 000 nodes in 1 second

XHProf~ 4 seconds for 50 nodes.~ 50% of the database time.~ 100% of the fields update time.

Anatomy of the node_save()field_attach_presave('node', $node);

module_invoke_all('node_presave', $node);

module_invoke_all('entity_presave', $node, 'node');

drupal_write_record('node', $node);

_node_save_revision($node, $user->uid);

node_invoke($node, $op); // $op == ‘update’ || $op == ‘insert’

-------------------------------------------------------------------------------------------

// Save fields.$function = "field_attach_$op";$function('node', $node); // field_attatch_update ||

field_attach_insert

module_invoke($storage_info['module'], 'field_storage_write', …..);

field_sql_storage_field_storage_write()

-------------------------------------------------------------------------------------------

module_invoke_all('node_' . $op, $node);module_invoke_all('entity_' . $op, $node, 'node');

node_access_acquire_grants($node, $delete);

entity_get_controller('node')->resetCache(array($node->nid));

You need all this if you want own fieldable entity.Procedural code is simple, isn’t it?

Long time ago in a galaxy far away...

Field data field & field data revision

foreach ($field['columns'] as $column => $attributes) {

$columns[] = _field_sql_storage_columnname($field_name, $column);

}

$query = db_insert($table_name)->fields($columns);

$revision_query = db_insert($revision_name)->fields($columns);

foreach ($field_languages as $langcode) {

$items = (array) $entity->{$field_name}[$langcode];

$delta_count = 0;

foreach ($items as $delta => $item) {

// We now know we have someting to insert. $do_insert = TRUE;

$record = array(

'entity_type' => $entity_type,

'entity_id' => $id,

'revision_id' => $vid,

'bundle' => $bundle,

'delta' => $delta,

'language' => $langcode,

);

foreach ($field['columns'] as $column => $attributes) {

$record[_field_sql_storage_columnname($field_name, $column)] = isset($item[$column]) ? $item[$column] : NULL;

}

$query->values($record);

if (isset($vid)) {

$revision_query->values($record);

}

if ($field['cardinality'] != FIELD_CARDINALITY_UNLIMITED && ++$delta_count == $field['cardinality']) {

break;

}

}

}

// Execute the query if we have values to insert. if ($do_insert) {

$query->execute();

$revision_query->execute();

}

}

Function loops for all fields in the entity end performs Delete/Insert

Can I haz some SQLs?

Story 2Back in my day...

Content Construction Kit: Drupal 6

Drupal 7

Why, Dries, why?https://drupal.org/node/1035804#comment-6860324

“This option is the one that "makes sense" to non-Drupal coders who have

written some PHP by hand before now and think they know their way around making their own database schema. It's not wrong if you are building one thing once..

Nowadays however, if you choose to use the Drupal framework and build re-usable tools,

• All Drupal text entity fields must be translatable (non-negotiable)

• All Drupal entity fields must be revisionable (non-negotiable)

• All Drupal entity fields must support being multiple (negotiable, but it's easier to allow them to be multiple as it's free to do so)

If the problem *you* are solving today doesn't actually seem to need all those features right now, that's quite likely. You may also know that you'll never need to do translations, and that a number of your values aren't going to need to be multiple (according to todays business rules anyway). Next year you'll probably wish you'd supported revisions, but that's next year.”

© dman

«I don’t need this»

• Field SQL Norevisions:

https://drupal.org/project/field_sql_norevisions

• MongoDB:

https://drupal.org/project/mongodb

• Per-Bundle Storage:

https://drupal.org/project/pbs

• EntityFieldQuery Views Backend:

https://drupal.org/project/efq_views

• Own storage?

Imagine

1. Storage level caching

2. Multi-update: node_save_multiple

3. Check if field was changed before

performing a query

4. Simple storage without delta and language

columns.

5. What else?

I’ll write own storage with a….

1. hook_field_storage_info().

2. hook_field_storage_details().

3. hook_field_storage_create_field().

4. hook_field_storage_update_field().

5. hook_field_storage_delete_field().

6. hook_field_storage_load().

7. hook_field_storage_write().

8. hook_entity_query_alter().

9. hook_field_storage_query().

10.hook_field_attach_rename_bundle().

11. hook_entity_insert().

12.hook_entity_update().

13.hook_field_attach_delete().

14.hook_entity_info_alter().

Let’s do it quick

http://posulliv.github.io/2013/01/07/bench-field-storage/

innodb_flush_log_at_trx_commit=0

innodb_doublewrite=0

log-bin=0

innodb_support_xa=0

innodb_buffer_pool_size=6G

innodb_log_file_size=512M

Curiosity is a way forward

Drupal 8

https://drupal.org/node/1497374

extends DatabaseStorageControllerNG

which extends DatabaseStorageController

which extends FieldableEntityStorageControllerBase

which implements FieldableEntityStorageControllerInterface

FieldableEntityStorageControllerBase

What’s next?

https://drupal.org/node/2083451

Reconsider the separate field revision data tables

https://drupal.org/node/2068325

[META] Convert entity SQL queries to the Entity Query API

Questions?