Storing User Files with Express, Stormpath, and Amazon S3

30
Storing User Files with Express, Stormpath, and Amazon S3 @rdegges

Transcript of Storing User Files with Express, Stormpath, and Amazon S3

Page 1: Storing User Files with Express, Stormpath, and Amazon S3

Storing User Files with Express,

Stormpath, and Amazon

S3

@rdegges

Page 2: Storing User Files with Express, Stormpath, and Amazon S3

I’m Randall Degges

Developer Evangelist at Stormpath

Python / Node / Go Hacker

Page 3: Storing User Files with Express, Stormpath, and Amazon S3

Agenda

- Stormpath Intro (5 minutes)

- Main Talk (30 minutes)- Q/A (15 minutes)

Page 4: Storing User Files with Express, Stormpath, and Amazon S3

Intro to Stormpath

API Service Website Mobile App

Page 5: Storing User Files with Express, Stormpath, and Amazon S3

We Do a Lot- Libraries for many languages- AD / LDAP- Single Sign On (SAML)- OAuth2- Social Login- Multi-Tenancy- Groups and Roles- Email Workflows

Page 6: Storing User Files with Express, Stormpath, and Amazon S3

What’s the problem?

Page 7: Storing User Files with Express, Stormpath, and Amazon S3

www.awesome.io

avatar.png

receipt.pdf

invoice.pdf

Stormpath!???

Page 8: Storing User Files with Express, Stormpath, and Amazon S3

How do people usually store user files?

Page 9: Storing User Files with Express, Stormpath, and Amazon S3

Database Columns

CREATE TABLE IF NOT EXISTS users ( id UUID DEFAULT uuid_generate_v4(), email TEXT PRIMARY KEY, password TEXT NOT NULL, avatar TEXT);

avatar.png

Base64 encode

Page 10: Storing User Files with Express, Stormpath, and Amazon S3

No!!!- Makes DB queries for each image view.

- Slows down DB.

- DBs aren’t good at this sort of IO (heavy disk reading).

- Slow performance for end users.

Page 11: Storing User Files with Express, Stormpath, and Amazon S3

Your Webserver(s)

Webserver

avatar.png

avatar.png

avatar.png

avatar.png

avatar.png

avatar.png

avatar.png

avatar.png

LOW

DI

SK

SPAC

E!!OMG!

Backups?

!

Page 12: Storing User Files with Express, Stormpath, and Amazon S3
Page 13: Storing User Files with Express, Stormpath, and Amazon S3

Amazon S3

- Reliability.- Durability.- Availability.- Cost.- Speed.- Security ACLs.

Page 14: Storing User Files with Express, Stormpath, and Amazon S3

How S3 Works

avatar.png

webserver

S3 Bucket

File URL

Custom Data (for user)

Page 15: Storing User Files with Express, Stormpath, and Amazon S3

User CustomData

{ "href": "https://api.stormpath.com/v1/accounts/gbMUL3uP8rFLZUMAw2XhI", "email": "[email protected]", "givenName": "Randall", "surname": "Degges", "customData": { "href": "https://api.stormpath.com/v1/accounts/gbMUL3uP8rFLZUMAw2XhI/customData", "s3": { "fluent.jpg": { "href": "https://s3.amazonaws.com/express-stormpath-s3/gbMUL3uP8rFLZUMAw2XhI/fluent.jpg", "lastModified": "2016-10-05T23:20:53.508Z" }, "wallhaven-204175.jpg": { "href": "https://s3.amazonaws.com/express-stormpath-s3/gbMUL3uP8rFLZUMAw2XhI/wallhaven-204175.jpg", "lastModified": "2016-10-06T01:08:58.898Z" } } }}

Page 16: Storing User Files with Express, Stormpath, and Amazon S3

Cool, Right?

Page 17: Storing User Files with Express, Stormpath, and Amazon S3

So… Let’s Build Something!

$ npm install express$ npm install express-stormpath$ npm install express-stormpath-s3

Page 18: Storing User Files with Express, Stormpath, and Amazon S3

"use strict";

const express = require("express");const stormpath = require("express-stormpath");const stormpathS3 = require("express-stormpath-s3");

let app = express();

app.use(stormpath.init(app));app.use(stormpath.getUser);

// Other middleware here// Routes here

app.listen(3000);

The Magic!

Page 19: Storing User Files with Express, Stormpath, and Amazon S3

/register

Page 20: Storing User Files with Express, Stormpath, and Amazon S3

/login

Page 21: Storing User Files with Express, Stormpath, and Amazon S3
Page 22: Storing User Files with Express, Stormpath, and Amazon S3
Page 23: Storing User Files with Express, Stormpath, and Amazon S3

app.use(stormpathS3({ awsAccessKeyId: process.env.AWS_ACCESS_KEY_ID, awsSecretAccessKey: process.env.AWS_SECRET_ACCESS_KEY, awsBucket: 'express-stormpath-s3'}));

This sets up the S3 client and adds simple file methods onto req.user. Required.

Page 24: Storing User Files with Express, Stormpath, and Amazon S3

Setup Done!

Page 25: Storing User Files with Express, Stormpath, and Amazon S3

Upload Files

req.user.uploadFile("./avatar.png", (err) => { if (err) throw err; console.log("Successfully uploaded file!");});

Path to local file you want to upload.

Page 26: Storing User Files with Express, Stormpath, and Amazon S3

Upload Files (cont)

req.user.uploadFile("./avatar.png", "public-read", (err) => { if (err) throw err; console.log("Successfully uploaded file!");});

Desired ACL for file.

Page 27: Storing User Files with Express, Stormpath, and Amazon S3

Delete Files

req.user.deleteFile("avatar.png", (err) => { if (err) throw err; console.log("Successfully deleted file!");});

Page 28: Storing User Files with Express, Stormpath, and Amazon S3

(demo)

Page 29: Storing User Files with Express, Stormpath, and Amazon S3

Resources- https://github.com/rdegges/express-stor

mpath-s3

- https://aws.amazon.com/s3/

- https://stormpath.com/

- https://github.com/rdegges/express-stormpath-s3-webinar

- https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html

Page 30: Storing User Files with Express, Stormpath, and Amazon S3

You’re awesome.

@rdegges