Cassandra Summit EU 2014 - Testing Cassandra Applications
-
Upload
christopher-batey -
Category
Software
-
view
1.351 -
download
0
Transcript of Cassandra Summit EU 2014 - Testing Cassandra Applications
chbatey
Cassandra is great - but how do I test it
chbatey
Who am Ibull Technical Evangelist for Apache Cassandra
bull Founder of Stubbed Cassandra
bull Help out Apache Cassandra users
bull Previous Senior software engineer BSkyB building Cassandra apps
chbatey
Overviewbull Cassandra overview
bull Cassandra failure scenarios
bull ReadTimeout amp WriteTimeout
bull Unavailable
bull Connectivity with the coordinator
bull Existing technologies
bull CCM
bull Cassandra Unit
bull Integration tests
bull Stubbed Cassandra
chbatey
Production
Application
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
replication factor 3
consistency ONE
chbatey
ProductionApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read and write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittle
bull Integration tests against an embedded Cassandra Eg cassandra-unit
bull Integration tests against an external Cassandra running on developer CI machines
bull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Who am Ibull Technical Evangelist for Apache Cassandra
bull Founder of Stubbed Cassandra
bull Help out Apache Cassandra users
bull Previous Senior software engineer BSkyB building Cassandra apps
chbatey
Overviewbull Cassandra overview
bull Cassandra failure scenarios
bull ReadTimeout amp WriteTimeout
bull Unavailable
bull Connectivity with the coordinator
bull Existing technologies
bull CCM
bull Cassandra Unit
bull Integration tests
bull Stubbed Cassandra
chbatey
Production
Application
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
replication factor 3
consistency ONE
chbatey
ProductionApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read and write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittle
bull Integration tests against an embedded Cassandra Eg cassandra-unit
bull Integration tests against an external Cassandra running on developer CI machines
bull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Overviewbull Cassandra overview
bull Cassandra failure scenarios
bull ReadTimeout amp WriteTimeout
bull Unavailable
bull Connectivity with the coordinator
bull Existing technologies
bull CCM
bull Cassandra Unit
bull Integration tests
bull Stubbed Cassandra
chbatey
Production
Application
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
replication factor 3
consistency ONE
chbatey
ProductionApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read and write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittle
bull Integration tests against an embedded Cassandra Eg cassandra-unit
bull Integration tests against an external Cassandra running on developer CI machines
bull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Production
Application
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
replication factor 3
consistency ONE
chbatey
ProductionApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read and write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittle
bull Integration tests against an embedded Cassandra Eg cassandra-unit
bull Integration tests against an external Cassandra running on developer CI machines
bull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Production
ApplicationApplicationApplicationApplicationApplication
replication factor 3
consistency ONE
chbatey
ProductionApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read and write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittle
bull Integration tests against an embedded Cassandra Eg cassandra-unit
bull Integration tests against an external Cassandra running on developer CI machines
bull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
ProductionApplicationApplicationApplicationApplicationApplication
ApplicationApplicationApplicationApplicationApplication
DC1
DC2
chbatey
Read and write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittle
bull Integration tests against an embedded Cassandra Eg cassandra-unit
bull Integration tests against an external Cassandra running on developer CI machines
bull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Read and write timeout
Application C
R1
R2
R3C=QUROUM
Replication factor 3
timeout
timeout
Read timeout
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittle
bull Integration tests against an embedded Cassandra Eg cassandra-unit
bull Integration tests against an external Cassandra running on developer CI machines
bull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Unavailable
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Unavailable
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittle
bull Integration tests against an embedded Cassandra Eg cassandra-unit
bull Integration tests against an external Cassandra running on developer CI machines
bull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
Set a socket read timeout
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittle
bull Integration tests against an embedded Cassandra Eg cassandra-unit
bull Integration tests against an external Cassandra running on developer CI machines
bull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Unit amp Integration testingbull Requirements Quick to run deterministic not brittle
bull Integration tests against an embedded Cassandra Eg cassandra-unit
bull Integration tests against an external Cassandra running on developer CI machines
bull Mocking the driver
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Acceptance testing
ApplicationAcceptance
test
prepare data
verification
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
How do this if it was a HTTP service
bull Release it - great book
bull Wiremock - mocking HTTP services
bull Saboteur - adding network latency
If you want to build a fault tolerant applicationyou better test faults
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Stubbed Cassandra
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Two sides of Scassandrabull Java
bullActs as a unitintegration testing library
bull90 of effort on this side
bull Standalone
bullRuns as a separate process
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Java Clientbull Embeds Stubbed Cassandra in a Java library
bull Start stop the server
bull Prime the server to return rows andor errors
bull Verify exactly the queries your application has executed
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Starting Stopping Scassandra scassandra = ScassandraFactorycreateServer()
scassandrastart()
PrimingClient primingClient = scassandraprimingClient()
ActivityClient activityClient = scassandraactivityClient()
ClassRulepublic static final ScassandraServerRule SCASSANDRA = new ScassandraServerRule()
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Activity Clientpublic ListltQuerygt retrieveQueries()
public ListltPreparedStatementExecutiongt retrievePreparedStatementExecutions()
public ListltConnectiongt retrieveConnections()
public void clearAllRecordedActivity() public void clearConnections()
public void clearQueries()
public void clearPreparedStatementExecutions()
bull Query Query text Consistency
bull PrepreparedStatementExecution Prepared statement text Bound variables
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Priming Clientpublic void prime(PrimingRequest prime) public ListltPrimingRequestgt retrievePreparedPrimes()public ListltPrimingRequestgt retrieveQueryPrimes()
public void clearAllPrimes()public void clearQueryPrimes()public void clearPreparedPrimes()
bull PrimingRequest Either a Query or PreparedStatement Query text or QueryPattern (regex) Consistency (default all) Result (success read timeout unavailable etc) Rows for successful response Column types for rows Variable types for prepared statements
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Example timepublic interface PersonDao void connect() void disconnect() ListltPersongt retrievePeople() ListltPersongt retrievePeopleByName(String firstName) void storePerson(Person person)
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Testing the connect methodTestpublic void shouldConnectToCassandraWhenConnectCalled() given activityClientclearConnections() when underTestconnect() then assertTrue(Expected at least one connection activityClientretrieveConnections()size() gt 0)
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Testing retrievepublic ListltPersongt retrieveNames() ResultSet result try Statement statement = new SimpleStatement(select from person) statementsetConsistencyLevel(ConsistencyLevelQUORUM) result = sessionexecute(statement) catch (ReadTimeoutException e) throw new UnableToRetrievePeopleException() ListltPersongt people = new ArrayListltgt() for (Row row result) peopleadd(new Person(rowgetString(first_name) rowgetInt(age))) return people
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Test the query + consistencyTestpublic void testQueryIssuedWithCorrectConsistencyUsingMatcher() given Query expectedQuery = Querybuilder() withQuery(select from person) withConsistency(QUORUM)build() when underTestretrievePeople() then assertThat(activityClientretrieveQueries() containsQuery(expectedQuery))
Hamcrest matcher
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Prepared statements
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Testing prepared statementTestpublic void testStorePerson() given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) build() primingClientprime(preparedStatementPrime) underTestconnect() when underTeststorePerson(new Person(Christopher 29)) then PreparedStatementExecution expectedPreparedStatement = PreparedStatementExecutionbuilder() withPreparedStatementText(insert into person(first_name age) values ()) withConsistency(ONE) withVariables(Christopher 29) build() assertThat(activityClientretrievePreparedStatementExecutions() preparedStatementRecorded(expectedPreparedStatement))
Hamcrest matcher for prepared statements
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Testing behaviourTestpublic void testRetrievingOfNames() throws Exception given MapltString gt singleRow = ImmutableMapof( first_name Chris last_name Batey age 29) MapltString ColumnTypesgt columnTypes = ImmutableMapof( age ColumnTypesInt ) primingClientprime(PrimingRequestqueryBuilder() withQuery(select from person) withColumnTypes(columnTypes) withRows(row) build()) ListltPersongt names = underTestretrievePeople() assertEquals(1 namessize()) assertEquals(Chris namesget(0)getName())
Each Cassandra Row == Java map
Tell Scassandra about your schema
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Testing errorsTest(expected = UnableToRetrievePeopleExceptionclass) public void testHandlingOfReadRequestTimeout() throws Exception given PrimingRequest primeReadRequestTimeout = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(primeReadRequestTimeout) when underTestretrievePeople() then
Expecting custom exception
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Testing retries Override public RetryDecision onReadTimeout(Statement statement ConsistencyLevel cl int requiredResponses int receivedResponses boolean dataRetrieved int nbRetry) if (nbRetry lt retries) return RetryDecisionretry(cl) else return RetryDecisionrethrow() Override public RetryDecision onWriteTimeout(Statement s ConsistencyLevel cl WriteType wt int requiredAcks int receivedAcks int nbRetry) return DefaultRetryPolicyINSTANCEonWriteTimeout(s cl wt receivedAcks receivedAcks nbRetry) Override public RetryDecision onUnavailable(Statement statement ConsistencyLevel cl int requiredReplica int aliveReplica int nbRetry) return DefaultRetryPolicyINSTANCEonUnavailable(statement cl requiredReplica aliveReplica nbRetry)
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Testing retriesTestpublic void testRetriesConfiguredNumberOfTimes() throws Exception PrimingRequest readTimeoutPrime = PrimingRequestqueryBuilder() withQuery(select from person) withResult(Resultread_request_timeout) build() primingClientprime(readTimeoutPrime) try underTestretrievePeople() catch (UnableToRetrievePeopleException e) assertEquals(CONFIGURED_RETRIES + 1 activityClientretrieveQueries()size())
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Coordinator issue
Application C
R1
R2
R3C=QUROUM
Replication factor 3
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Testing slow connectionTest(expected = UnableToSavePersonExceptionclass) public void testThatSlowQueriesTimeout() throws Exception given PrimingRequest preparedStatementPrime = PrimingRequestpreparedStatementBuilder() withQuery(insert into person(first_name age) values ()) withVariableTypes(ColumnTypesVarchar ColumnTypesInt) withFixedDelay(1000) build() primingClientprime(preparedStatementPrime) underTestconnect() underTeststorePerson(new Person(Christopher 29)) Delays the response by 1000ms
Expect a custom exception
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Testing slow connectionSocketOptions socketOptions = new SocketOptions()socketOptionssetReadTimeoutMillis(500)cluster = Clusterbuilder() addContactPoint(localhost) withPort(port) withRetryPolicy(new RetryReads()) withSocketOptions(socketOptions) build()
Set a read timeout
Overridepublic void storePerson(Person person) try BoundStatement bind = storeStatementbind(persongetName() persongetAge()) sessionexecute(bind) catch (NoHostAvailableException e) throw new UnableToSavePersonException()
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Summary of features Start as many instances as you like in the same JVM
Prime all primitive types
Prime sets and lists of type text
Prime prepared statements
Verify number of connections your application makes
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Future features Loading of schema for priming prepared statements
Pretending to be multiple nodes
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
How does this work Server implemented in Scala
Binary port for your Cassandra application to connect to
Admin port for the REST API
Thin Java wrapper to make it easy to be used in JUnit tests
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
How do I get itbull It is on maven central
bullOr go to wwwscassandraorg
bullExecutable jar + REST API
ltdependencygt ltgroupIdgtorgscassandraltgroupIdgt ltartifactIdgtjava-clientltartifactIdgt ltversiongt040ltversiongt ltscopegttestltscopegt ltdependencygt
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
I want to help Server httpsgithubcomscassandrascassandra-server
Client httpsgithubcomscassandrascassandra-java-client
Tests httpsgithubcomscassandrascassandra-it-java-driver-2
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Summary Testing is good - I want more tools to help me
Existing tools for Cassandra are great at happy path scenarios
Stubbed Cassandra allows testing of edge cases
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
The end - Questions
wwwscassandraorg
httpchristopher-bateyblogspotcouk
chbatey
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
But I donrsquot like Java
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Separate process
ApplicationAcceptance
test
prime on admin port (REST)
verification on admin port
Admin endpoints
Native protocol
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
The REST API - priming when query select from person consistency [ONE TWO]
then rows [first_name Chris last_name Batey
first_name Mansoor last_name Panda
first_name Wayne last_name Rooney] column_types last_name text
result success
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
The REST API - Activity[ query SELECT FROM systemschema_columns consistency ONE query SELECT FROM systempeers consistency ONE query SELECT FROM systemlocal WHERE key=local consistency ONE query use people consistency ONE query select from people consistency TWO ]
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Priming prepared statements when queryinsert into people(first_name last_name) values () then variable_types[ ascii text ]
result success
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Priming errors when query select from people where name = then result read_request_timeout
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set
chbatey
Priming cluster name when query SELECT FROM systemlocal WHERE key=local then rows [ cluster_name custom cluster name partitioner ldquoorgapachecassandradhtMurmur3Partitioner data_center dc1 rack rc1 tokens [ 1743244960790844724] release_version 201 ] result success column_types tokens set