Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked...

35
1/ Libsmb2: a Userspace SMB2 Client for all Platforms Ronnie Sahlberg, RedHat Storage Developer Conference 2019

Transcript of Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked...

Page 1: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

1 /

Libsmb2: a Userspace SMB2 Client for all Platforms

Ronnie Sahlberg, RedHat

Storage Developer Conference 2019

Page 2: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

2 /

Legal statement

●This work represents the views of the author(s) and does not necessarily reflect the views of IBM/RedHat

●This is a personal project, not a RedHat project

●Any complaints about this project should be directed at me

Storage Developer Conference 2019

Page 3: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

3 /

Who Am I?

●Ronnie Sahlberg, Redhat

●Worked with network storage protocols for a long time.

●Wireshark, Samba, CTDB, libnfs, libiscsi

●And now LIBSMB2 !

Storage Developer Conference 2019

Page 4: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

4 /

Prelude

●In this talk I will cover libsmb2. What it is and why you should use it.

●We will cover the main features, look at some code and run some quick demos.

●I hope I will convert at least some of you to try it/use it in your projects.

Storage Developer Conference 2019

Page 5: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

5 /

What is libsmb2

●Libsmb2 is a small footprint SMB2/3 client that runs in userspace.

●Libsmb2 is highly portable. It builds and runs on Windows/Linux/BSD/MacOS/iOS/Android/WatchOS/… and even on some micro-controllers.

●It also builds on UWP so if you want that XBOX application you are building to have SMB2/3 access ...

Storage Developer Conference 2019

Page 6: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

6 /

What is libsmb2

●Libsmb2 is a small footprint SMB2/3 client that runs in userspace.

●Libsmb2 is highly portable. It builds and runs on Windows/Linux/BSD/MacOS/iOS/Android/WatchOS/… and even on some micro-controllers.

●It also builds on UWP so if you want that XBOX application you are building to have SMB2/3 access ...

Storage Developer Conference 2019

Page 7: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

7 /

What is libsmb2

●It is fast, and have NO other dependencies than a basic libc (unless you want Kerberos)

●It is zero-copy, even for reads.(except for encryption)

Storage Developer Conference 2019

Page 8: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

8 /

Licence

●It is licenced under LGPLv2.1 (or later) which means it is app-store-friendly and is compatible with the distribution model for all main platforms, including wearable and mobile (afaik).

●If you have a problem with the license, please talk to me and we can see what we can work out. (There is empirical evidence I WILL go the extra mile to try to help you fix any licensing concerns.)

Storage Developer Conference 2019

Page 9: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

9 /

EventDriven not threaded

●Libsmb2 is an event driven design and is fully non-blocking.

●It is NOT multithread safe but that is trivial to add in your application, if you need it.

Storage Developer Conference 2019

Page 10: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

10 /

Event API

Just like libnfs, you have a simple API to interface the library with your event system.

●smb2_get_fd()

●smb2_which_events()

●smb2_service()

Storage Developer Conference 2019

Page 11: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

11 /

Event API

Storage Developer Conference 2019

while (!is_finished) {pfd.fd = smb2_get_fd(smb2);pfd.events = smb2_which_events(smb2);

if (poll(&pfd, 1, 1000) < 0) {printf("Poll failed");break;

}if (pfd.revents == 0) {

continue;}

if (smb2_service(smb2, pfd.revents) < 0) {printf("smb2_service failed with : %s\n",

smb2_get_error(smb2));break;

}}

Page 12: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

12 /

Three different APIs

●High-level async API with posix-like functions, i.e. smb2_open_async()

●High-level sync API with posix-like functions, i.e. smb2_open(). The sync interface is meant only for trivial apps. Don’t use it for real/complex apps.

●Low level async API that provides access to raw SMB2 commands.

Storage Developer Conference 2019

Page 13: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

13 /

DEMO

… look at example code smb2-cat-[a]sync ...

Storage Developer Conference 2019

Page 14: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

14 /

URL syntax

smb://[<domain>;][<user>@]<server>[:<port>]/<share>[/path][?arg=val[&arg=val]*]

●I do NOT support setting a password from the URL/command line.

●You either use NTLM_USER_FILE or smb2_set_password()

Page 15: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

15 /

Authentication

●Supports Guest and Anonymous

●Built-in NTLMSSP authentication (could not find any existing libraries with a suitable licence)

●Can do Kerberos to AD IFF we link with external MIT-Kerberos libraries. This is THE ONLY external dependency we have and it is optional.

●May need to write my own KRB5 client :-(

Storage Developer Conference 2019

Page 16: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

16 /

Crypto

●We have our own set of crypto primitives we need.

●(Used to depend on OPENSSL but we don’t do that any more.)

Storage Developer Conference 2019

Page 17: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

17 /

Signing

●Both SMB2 and 3 signing is supported.

Storage Developer Conference 2019

Page 18: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

18 /

Encryption

●SMB3 encryption is supported.

●Fun fact: AES128CCM object code is only ~2kb

Storage Developer Conference 2019

Page 19: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

19 /

DEMO

… show smb2-ls-async with encryption …… and wireshark ...

Storage Developer Conference 2019

Page 20: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

20 /

DCE/RPC

●We have a small, rudimentary DCE/RPC runtime. It is very limited.

●It only handles IPC$ named pipes as transport.

●But it supports both NDR and NDR64

●No IDL compiler

Storage Developer Conference 2019

Page 21: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

21 /

DCE/RPC

●The DCE/RPC implementation is really only to provide NetShareEnum.

●But it is simple enough to use that writing the encoders/decoders by hand is viable, for now.

... show dcerpc-srvsvc.c and talk ...

Storage Developer Conference 2019

Page 22: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

22 /

Size

Storage Developer Conference 2019

Small is beautiful

Page 23: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

23 /

Libsmb2 is SMALL

●Estimating the size by stripping the binaries

●The whole library itself is ~140kb

… show the stripped binaries ...

Storage Developer Conference 2019

Page 24: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

24 /

Can we make is smaller?

●If we drop DCE/RPC (NetShareEnum) we can claw back ~11.5kb

●If we drop crypto (authentication, sign, seal) we can claw back another ~37kb

Make it more modular so we can disable certain features at buildtime to make it even smaller?

Storage Developer Conference 2019

Page 25: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

25 /

Small enough for IoT, uC

●But even the full library, at ~140kb, should be small enough it can fit your tiny IoT devices.

Storage Developer Conference 2019

Page 26: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

26 /

User feedback, VLC

Storage Developer Conference 2019

"SMB support on Android and iOS was number one request since a very long time. We could not use Sam

We wrote a simple library for SMBv1, called libDSM, and it worked as a stopgap, for some time. But as SM

– Jean-Baptiste Kempf

"VLC use libsmb2 principally for mobile ports (Android/iOS) and macOS since these ports can't use

– Thomas Guillem

Page 27: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

27 /

User feedback, Kodi

Storage Developer Conference 2019

Main purpose to step into using your library was adding support of reading/browsing shares on our K

– Anton Fedchin

Page 28: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

28 /

Coolest project ever

Storage Developer Conference 2019

Thanks again for doing libsmb2 and for your help getting it to run on an ESP32.

– Arno Puder

Page 29: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

29 /

ESP32

●www.espressif.com

●It has 520kb RAM. That is very small but enough to fit an application, a network stack and libsmb2, with room to spare.

Storage Developer Conference 2019

Page 30: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

30 /

TRS-80

●Check out https://github.com/apuder/TRS-IO

●(show it in browser)

●There is a TRS-80 out there that that can access files via SMB2!

Storage Developer Conference 2019

Page 31: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

31 /

But where is it

●https://github.com/sahlberg/libsmb2

Storage Developer Conference 2019

Page 32: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

32 /

How to contribute?

●Pull requests

●Patch email to me : [email protected]

●Email with a description of a fix

●Report a bug

This is one of the easiest and lowest threshold projects to contribute to.

Storage Developer Conference 2019

(any way you want!)

Page 33: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

33 /

Why should you use it

●License

●Portability

●Size

●Coolness

Storage Developer Conference 2019

(instead of something else?)

Page 34: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

34 /

Future roadmap?

●You tell me. I think it is pretty feature complete at this point.

●Add SMB3.1.1.

●Make it more modular to strip off/disable features to save size.

●Built-in krb5 client?

Storage Developer Conference 2019

Page 35: Libsmb2: a Userspace SMB2 Client for all Platforms...3 / Who Am I? Ronnie Sahlberg, Redhat Worked with network storage protocols for a long time. Wireshark, Samba, CTDB, libnfs, libiscsi

35 /

Q n A

Storage Developer Conference 2019

Questions?