Mateusz herych content search problem on android
-
Upload
apps4allru -
Category
Software
-
view
165 -
download
0
description
Transcript of Mateusz herych content search problem on android
![Page 1: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/1.jpg)
![Page 2: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/2.jpg)
Mateusz Herych
Android Developer - Base CRMCo-organizer - GDG KrakowCo-organizer - KrakDroid
![Page 3: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/3.jpg)
Stats
![Page 4: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/4.jpg)
LIKE ‘%smth%’
![Page 5: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/5.jpg)
LIKE ‘%smth%’is not the way.
![Page 6: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/6.jpg)
Search
![Page 7: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/7.jpg)
SearchOffline.
![Page 8: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/8.jpg)
Why?
![Page 9: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/9.jpg)
Why?Let the backend guys do the job
![Page 10: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/10.jpg)
Why?Internet is not everywhere.
![Page 11: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/11.jpg)
Why?Internet is not everywhere.It takes time. (especially SSL)
![Page 12: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/12.jpg)
Why?Internet is not everywhere.It takes time. (especially SSL)And sometimes it’s shitty.
![Page 13: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/13.jpg)
Why?Internet is not everywhere.It takes time. (especially SSL)And sometimes it’s shitty.
![Page 14: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/14.jpg)
Sure, some apps don’t really need it
You need an Internet to order that taxi anyway
![Page 15: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/15.jpg)
Do you keep offline content?
Let your users navigate fast.
![Page 16: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/16.jpg)
Did I say fast?
![Page 17: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/17.jpg)
How?Let’s go deeper.
![Page 18: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/18.jpg)
Context
![Page 19: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/19.jpg)
CRM- Contacts- Deals- Notes- ...
![Page 20: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/20.jpg)
CRM- Contacts (~100)- Deals (~50)- Notes (~100)- ... 2011
![Page 21: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/21.jpg)
select id from deals where name LIKE ‘%something%’
![Page 22: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/22.jpg)
CRM- Contacts (~40K)- Deals (~20K)- Notes (~300K)- ...
![Page 23: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/23.jpg)
![Page 24: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/24.jpg)
HOW DOES “LIKE” WORKS LIKE?
![Page 25: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/25.jpg)
Docs saying
![Page 26: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/26.jpg)
I tried to put all the conditions that need to be satisfied so SQLite can use indices combined with LIKE
operator.
Docs saying
![Page 27: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/27.jpg)
They didn’t fit.
Docs saying
![Page 28: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/28.jpg)
http://www.sqlite.org/optoverview.html
Docs saying
![Page 29: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/29.jpg)
Hey, you, SQLite!
EXPLAIN (my) QUERY PLAN
![Page 30: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/30.jpg)
PRAGMA case_sensitive_like=1;
![Page 31: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/31.jpg)
PRAGMA case_sensitive_like=1;CREATE INDEX search_index on deals(name);
![Page 32: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/32.jpg)
PRAGMA case_sensitive_like=1;CREATE INDEX search_index on deals(name);SELECT id FROM deals WHERE name LIKE ‘Some%’;
![Page 33: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/33.jpg)
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE ‘Some%’;
SEARCH TABLE deals USING COVERING INDEX search_index (name>? AND name<?) (~31250 rows)
![Page 34: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/34.jpg)
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE ‘%Some%’;
![Page 35: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/35.jpg)
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE ‘%Some%’;
SCAN TABLE deals (~500000 rows)
![Page 36: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/36.jpg)
EXPLAIN QUERY PLAN SELECT id FROM deals WHERE name LIKE ‘%Some%’;
SCAN TABLE deals (~500000 rows)
(And then you die)
![Page 37: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/37.jpg)
first_name || ‘ ‘ || last_name?complicated queries, VIEWs?
Like is NOT the way to go.
![Page 38: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/38.jpg)
What people think SQLite is
![Page 39: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/39.jpg)
What SQLite really is
![Page 40: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/40.jpg)
SQLite is powerful
Not kidding.
![Page 41: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/41.jpg)
FTS3Full Text Search
![Page 42: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/42.jpg)
CREATE VIRTUAL TABLE search USING fts3 (tokens)
![Page 43: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/43.jpg)
?
CREATE VIRTUAL TABLE search USING fts3 (tokens INT)
![Page 44: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/44.jpg)
Nope.
PRAGMA table_info(search);cid|name|type|notnull|dflt_value|pk0|word||0||0
![Page 45: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/45.jpg)
All is TEXT, except for hidden rowid.
![Page 46: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/46.jpg)
What is virtual table?Imagine it’s a Java interface.interface VirtualTable { void insert(Params p); void update(Params p); // etc, also createTable.}
![Page 47: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/47.jpg)
What is a virtual table?
class Fts3 implements VirtualTable { // …}
![Page 48: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/48.jpg)
![Page 49: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/49.jpg)
MATCHLet’s go make some magic.
![Page 50: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/50.jpg)
SELECT * FROM search WHERE content MATCH ‘something’
![Page 51: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/51.jpg)
SELECT rowid, * FROM search WHERE content MATCH ‘something’rowid|word1|something2|not something special3|SoMeThInG
![Page 52: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/52.jpg)
SELECT rowid, * FROM search WHERE contentMATCH ‘some* spe*’
rowid|word2|not something special
![Page 53: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/53.jpg)
CREATE VIRTUAL TABLE search USING fts3 (author, lyrics)
![Page 54: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/54.jpg)
SELECT * FROM search WHERE lyrics MATCH ‘author:Giorgio Synthesizer
author |lyricsGiorgio Moroder|..Why don’t I use a synthesizer...
![Page 55: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/55.jpg)
Cool?
![Page 56: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/56.jpg)
Cool?Look at this.
![Page 57: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/57.jpg)
SELECT * FROM search WHERE lyrics MATCH ‘why NEAR synthesizer’
author |lyricsGiorgio Moroder|..Why don’t I use synthesizer...
![Page 58: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/58.jpg)
SELECT * FROM search WHERE lyrics MATCH ‘why NEAR/3 synthesizer’
author |lyricsGiorgio Moroder|..Why don’t I use synthesizer...
![Page 59: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/59.jpg)
Tips.
![Page 60: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/60.jpg)
1. Your FTS vtable should contain only tokens. Divided into sections.
![Page 61: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/61.jpg)
2. Link your FTS table’s records with other table (containing real object’s id and type) using rowid.
![Page 62: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/62.jpg)
3. Remember. FTS is fast enough for searching purposes. But it’s always slower than ‘=’ based query on indexed field.
![Page 63: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/63.jpg)
4. EXPLAIN QUERY PLAN doesn’t work for fts tables. Try to measure it with .timer ON.
![Page 64: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/64.jpg)
5. ???
![Page 65: Mateusz herych content search problem on android](https://reader033.fdocuments.in/reader033/viewer/2022051513/547c50f0b37959822b8b4f30/html5/thumbnails/65.jpg)
6. QUESTIONS TIME!