MODULES ARE COMING...1 Bryce Adelstein Lelbach CUDA C++ Core Libraries Lead ISO C++ Library...

Post on 12-Mar-2020

7 views 0 download

Transcript of MODULES ARE COMING...1 Bryce Adelstein Lelbach CUDA C++ Core Libraries Lead ISO C++ Library...

1

Bryce Adelstein Lelbach

CUDA C++ Core Libraries Lead

ISO C++ Library Evolution Incubator Chair, ISO C++ Tooling Study Group Chair

MODULES ARE COMING@blelbach

22Copyright (C) 2019 Bryce Adelstein Lelbach

3

C++20 will have as big an impact as C++11.

• Concepts.

• Coroutines

• Improved constexpr.

• Ranges.

• Modules.

3Copyright (C) 2019 Bryce Adelstein Lelbach

4

Modules are:

• A new compilation model for C++.

• A new way to organize C++ projects.

4Copyright (C) 2019 Bryce Adelstein Lelbach

5

#include “math.hpp”

int main() { return square(42); }

main.cpp

#include “math.hpp”

int square(int a) { return a * a; }

math.cpp

#pragma once

int square(int a);

math.hpp

Textual Inclusion

5Copyright (C) 2019 Bryce Adelstein Lelbach

6

#include “math.hpp”

int main() { return square(42); }

main.cpp

#include “math.hpp”

int square(int a) { return a * a; }

math.cpp

#pragma once

int square(int a);

math.hpp

Modular Import

import math;

int main() { return square(42); }

main.cpp

module math;

int square(int a) { return a * a; }

math.mxx

export module math;

export int square(int a);

math.ixx

Textual Inclusion

6Copyright (C) 2019 Bryce Adelstein Lelbach

7

Modules will have a greater impact than any other feature added post C++-98.

7Copyright (C) 2019 Bryce Adelstein Lelbach

8

… C++ code is written

C++11’s <thread> changes how…

8Copyright (C) 2019 Bryce Adelstein Lelbach

9

… C++ code is written

C++11’s lambdas change how…

9Copyright (C) 2019 Bryce Adelstein Lelbach

10

… C++ code is written

… C++ code is parsed

C++11’s lambdas change how…

10Copyright (C) 2019 Bryce Adelstein Lelbach

11

… C++ code is written

C++20’s modules change how…

11Copyright (C) 2019 Bryce Adelstein Lelbach

12

… C++ code is written

… C++ code is parsed

… C++ projects are consumed

C++20’s modules change how…

12Copyright (C) 2019 Bryce Adelstein Lelbach

13

… C++ code is written

… C++ code is parsed

… C++ projects are compiled

C++20’s modules change how…

13Copyright (C) 2019 Bryce Adelstein Lelbach

14

… C++ code is written

… C++ code is parsed

… C++ projects are compiled

… C++ projects are organized

C++20’s modules change how…

14Copyright (C) 2019 Bryce Adelstein Lelbach

15

… C++ code is written

… C++ code is parsed

… C++ projects are compiled

… C++ projects are organized

… C++ projects are consumed

C++20’s modules change how…

15Copyright (C) 2019 Bryce Adelstein Lelbach

16

Modules will have a greater impact than any other feature added post C++-98.

16Copyright (C) 2019 Bryce Adelstein Lelbach

17

Today’s Compilation Model17Copyright (C) 2019 Bryce Adelstein Lelbach

18

What is C++’s compilation model today?

How do we organize C++ projects today?

18Copyright (C) 2019 Bryce Adelstein Lelbach

19

“The text of the program is kept in units called source files

in this International Standard.”

[lex.separate] p1 s1

19Copyright (C) 2019 Bryce Adelstein Lelbach

20

“A source file together with all the headers and source filesincluded via the preprocessing directive #include, less any

source lines skipped by any of the conditional inclusion

preprocessing directives, is called a translation unit.”

[lex.separate] p1 s2

20Copyright (C) 2019 Bryce Adelstein Lelbach

21

Example Extension Artifact Notes

Non-Modular Unit#include “…”…

.cpp .o

Kinds of Translation Units

21Copyright (C) 2019 Bryce Adelstein Lelbach

22

“A program consists of one or more translation units linked

together.”

[basic.link] p1 s1

22Copyright (C) 2019 Bryce Adelstein Lelbach

23

“Previously translated translation units and instantiation

units can be preserved individually or in libraries.”

[lex.separate] p2 s1

23Copyright (C) 2019 Bryce Adelstein Lelbach

24

Textual Inclusion: Preprocess

a.cpp

#include “a.hpp”#include “b.hpp”#include “c.hpp”

b.cpp

#include “c.hpp”#include “b.hpp”#include “lib.hpp”

a.hpp b.hpp c.hpp lib.hpp

lib.so

24Copyright (C) 2019 Bryce Adelstein Lelbach

25

Textual Inclusion: Compile

a.cpp (TU) b.cpp (TU)

a.hpp

b.hpp

c.hpp

b.hpp

b.hpp

c.hpp

lib.so

25Copyright (C) 2019 Bryce Adelstein Lelbach

26

Textual Inclusion: Link

a.o (TU) b.o (TU)

lib.so

26Copyright (C) 2019 Bryce Adelstein Lelbach

27

Textual Inclusion

lib.so

exe

27Copyright (C) 2019 Bryce Adelstein Lelbach

28

lib.so

exe

a.o (TU) b.o (TU)

lib.so

a.cpp (TU) b.cpp (TU)

a.hpp

b.hpp

c.hpp

b.hpp

b.hpp

c.hpp

lib.so

a.cpp b.cpp

a.hpp b.hpp c.hpp lib.hpp

lib.so

Preprocess Compile Link

Textual Inclusion

28Copyright (C) 2019 Bryce Adelstein Lelbach

29

a.o: a.cpp$(CC) -c a.cpp -o a.o

b.o: b.cpp$(CC) -c b.cpp -o b.o

exe: a.o b.o$(CC) a.o b.o lib.so -o exe

29Copyright (C) 2019 Bryce Adelstein Lelbach

30

Headers are terrible:

• Slow to compile.

• ODR violations.

• Lack of encapsulation.

• Cyclic dependencies.

• Order dependent.

30Copyright (C) 2019 Bryce Adelstein Lelbach

31

Headers are terrible:

• Slow to compile.

• ODR violations.

• Lack of encapsulation.

• Cyclic dependencies.

• Order dependent.

31Copyright (C) 2019 Bryce Adelstein Lelbach

32

a.cpp (TU) b.cpp (TU)

a.hpp

b.hpp

c.hpp

b.hpp

b.hpp

c.hpp

Textual Inclusion: Compile

lib.so

32Copyright (C) 2019 Bryce Adelstein Lelbach

33

0.cpp 1.cpp 2.cpp 3.cpp 4.cpp 5.cpp 6.cpp

a.hpp b.hpp c.hpp

Pro: Embarrassingly parallel.

Con: a.hpp, b.hpp, and c.hpp are compiled 7 times.

Textual Inclusion

33Copyright (C) 2019 Bryce Adelstein Lelbach

34

Headers are terrible:

• Slow to compile.

• ODR violations.

• Lack of encapsulation.

• Cyclic dependencies.

• Order dependent.

34Copyright (C) 2019 Bryce Adelstein Lelbach

35

“There can be more than one definition of [an entity] in a

program provided that no prior definition is necessarily

reachable at the point where a definition appears, and

provided the definitions satisfy the following requirements …”

[basic.def.odr] p12

35Copyright (C) 2019 Bryce Adelstein Lelbach

36

Ill formed, no diagnostic required

(IFNDR)

36Copyright (C) 2019 Bryce Adelstein Lelbach

37

#define DEBUG#include “tree_node.hpp”

// …

a.cpp

#include “tree_node.hpp”

// …

b.cpp

#pragma once

template <typename T>struct tree_node {T value;std::vector<tree_node*> children;#ifdef DEBUGtree_node* parent;#endif};

tree_node.hpp

37Copyright (C) 2019 Bryce Adelstein Lelbach

38

Headers are terrible:

• Slow to compile.

• ODR violations.

• Lack of encapsulation.

• Cyclic dependencies.

• Order dependent.

38Copyright (C) 2019 Bryce Adelstein Lelbach

39

#pragma once

namespace c {struct A {private:template <typename T>void impl();

};

namespace detail::unsupported {template <typename T>void __please_dont_use();}}

a.hpp

39Copyright (C) 2019 Bryce Adelstein Lelbach

40

Headers are terrible:

• Slow to compile.

• ODR violations.

• Lack of encapsulation.

• Cyclic dependencies.

• Order dependent.

40Copyright (C) 2019 Bryce Adelstein Lelbach

41

#pragma once#include “a.hpp”

struct X;struct Y { X* x; };

b.hpp

#pragma once#include “b.hpp”

struct Y;struct X { Y* y; };

a.hpp

41Copyright (C) 2019 Bryce Adelstein Lelbach

42

Headers are terrible:

• Slow to compile.

• ODR violations.

• Lack of encapsulation.

• Cyclic dependencies.

• Order dependent.

42Copyright (C) 2019 Bryce Adelstein Lelbach

43

#pragma once

void foo(S s);

b.hpp

#pragma once

struct S { /* … */ };

a.hpp

#include “a.hpp”#include “b.hpp”

c.cpp

43Copyright (C) 2019 Bryce Adelstein Lelbach

44

#pragma once

void foo(S s);

b.hpp

#pragma once

struct S { /* … */ };

a.hpp

#include “b.hpp”#include “a.hpp”

d.cpp

#include “a.hpp”#include “b.hpp”

c.cpp

44Copyright (C) 2019 Bryce Adelstein Lelbach

45

Headers are terrible:

• Slow to compile.

• ODR violations.

• Lack of encapsulation.

• Cyclic dependencies.

• Order dependent.

45Copyright (C) 2019 Bryce Adelstein Lelbach

46

Using Modules46Copyright (C) 2019 Bryce Adelstein Lelbach

47

#include “foo.hpp”

Textual Inclusion

Modular Import

import foo;

47Copyright (C) 2019 Bryce Adelstein Lelbach

48

a.mxx

import b;import c;

b.mxx

import c;import lib;

a.ixx b.ixx c.ixx lib.ixx

lib.so

Modular Import: Precompile

48Copyright (C) 2019 Bryce Adelstein Lelbach

49

a.mxx

import b;import c;

b.mxx

import c;import lib;

a.ixx b.ixx c.ixx lib.ixx

lib.cmi

lib.so

c.cmib.cmia.cmi

Modular Import: Preprocess

49Copyright (C) 2019 Bryce Adelstein Lelbach

50

a.mxx (TU) b.mxx (TU)

lib.cmi

lib.so

c.cmib.cmia.cmi

Modular Import: Compile

50Copyright (C) 2019 Bryce Adelstein Lelbach

51

a.m.o (TU) b.m.o (TU)

lib.i.o

lib.so

c.i.ob.i.oa.i.o

Modular Import: Link

51Copyright (C) 2019 Bryce Adelstein Lelbach

52

Modular Import

exe

lib.so

52Copyright (C) 2019 Bryce Adelstein Lelbach

53

a.mxx b.mxx

a.ixx b.ixx c.ixx lib.ixx

lib.so

exe

lib.so

a.m.o (TU) b.m.o (TU)

lib.i.o

lib.so

c.i.ob.i.oa.i.o

a.mxx (TU) b.mxx (TU)

lib.cmi

lib.so

c.cmib.cmia.cmi

a.mxx b.mxx

a.ixx b.ixx c.ixx lib.ixx

lib.cmi

lib.so

c.cmib.cmia.cmi

Precompile Preprocess Compile Link

lib.so

exe

a.o (TU) b.o (TU)

lib.so

a.cpp (TU) b.cpp (TU)

a.hpp

b.hpp

c.hpp

b.hpp

b.hpp

c.hpp

lib.so

a.cpp b.cpp

a.hpp b.hpp c.hpp lib.hpp

lib.so

Preprocess Compile Link

Modular Import

Textual Inclusion

53Copyright (C) 2019 Bryce Adelstein Lelbach

54

Module Precompilation

a.cmi: a.ixx$(CC) --precompile a.ixx -o a.cmi

b.cmi: b.ixx$(CC) --precompile b.ixx -o b.cmi

c.cmi: c.ixx$(CC) --precompile c.ixx -o c.cmi

lib.cmi: lib.ixx$(CC) --precompile lib.ixx -o lib.cmi

54Copyright (C) 2019 Bryce Adelstein Lelbach

55

Module Interface Unit Compilation

a.i.o: a.cmi$(CC) -c a.cmi -o a.i.o

b.i.o: b.cmi$(CC) -c b.cmi -o b.i.o

c.i.o: c.cmi$(CC) -c c.cmi -o c.i.o

lib.i.o: lib.cmi$(CC) -c lib.cmi -o lib.i.o

55Copyright (C) 2019 Bryce Adelstein Lelbach

56

a.m.o: a.mxx a.cmi b.cmi c.cmi$(CC) -c a.mxx -o a.m.o

b.m.o: b.mxx b.cmi c.cmi lib.cmi$(CC) -c b.mxx -o b.m.o

exe: a.m.o b.m.o a.i.o b.i.o c.i.o lib.i.o$(CC) a.o b.o a.i.o b.i.o c.i.o lib.i.o lib.so -o exe

Compilation and Link

56Copyright (C) 2019 Bryce Adelstein Lelbach

57

0.cpp 1.cpp 2.cpp 3.cpp 4.cpp 5.cpp 6.cpp

a.hpp b.hpp c.hpp

Pro: Embarrassingly parallel.

Con: a.hpp, b.hpp, and c.hpp are compiled 7 times.

Textual Inclusion

57Copyright (C) 2019 Bryce Adelstein Lelbach

58

a.ixx b.ixx c.ixx

0.cpp 1.cpp 2.cpp 3.cpp 4.cpp 5.cpp 6.cpp

a.cmi b.cmi c.cmi

Modular Inclusion

Pro: a.ixx, b.ixx, and c.ixx are precompiled once.

Con: Not embarrassingly parallel.58Copyright (C) 2019 Bryce Adelstein Lelbach

59

Example Extension Artifact Notes

Non-Modular Unit#include “…”…

.cpp .o

Module Interface Unitexport module …;…

.ixx.cmi.o (optional)

Exactly one per module.

Module Implementation Unitmodule …;…

.mxx .o

Kinds of Translation Units

59Copyright (C) 2019 Bryce Adelstein Lelbach

60

#include “math.hpp”

int main() { return square(42); }

main.cpp

#include “math.hpp”

int square(int a) { return a * a; }

math.cpp

#pragma once

int square(int a);

math.hpp

Modular Import

import math;

int main() { return square(42); }

main.cpp

module math;

int square(int a) { return a * a; }

math.mxx

export module math;

export int square(int a);

math.ixx

Textual Inclusion

60Copyright (C) 2019 Bryce Adelstein Lelbach

61

import math;

int main() { return square(42); }

main.cpp

export module math;

export template <typename T>T square(T a) { return a * a; }

math.ixx

Modular Import

import square;

int main() { return square(42); }

main.cpp

#pragma once

template <typename T>T square(T a) { return a * a; }

math.hpp

Textual Inclusion

61Copyright (C) 2019 Bryce Adelstein Lelbach

62

#include <foo.hpp>#include “foo.hpp”

Textual Inclusion

Modular Import

import foo;import <foo.hpp>;import “foo.hpp”;

62Copyright (C) 2019 Bryce Adelstein Lelbach

63

“An importable header is a member of an implementation-defined set of

headers, that includes all importable C++ library headers.”

[module.unit] p5 s3 (with P1502)

63Copyright (C) 2019 Bryce Adelstein Lelbach

64

“An importable header is a member of an implementation-defined set of

headers, that includes all importable C++ library headers.”

[module.unit] p5 s3 (with P1502)

“A module-import-declaration that specifies a header-name H imports a

synthesized header unit … H shall identify an importable header.”

[module.unit] p5 s1, s4

64Copyright (C) 2019 Bryce Adelstein Lelbach

65

Importable headers:

• Most C++ standard library headers*.

• Some system headers.

• Headers you proclaim importable†.

* C standard library headers (<cfoo>, <foo.h>) are not required to be importable.

† The mechanism for indicating which headers are importable is implementation defined.

65Copyright (C) 2019 Bryce Adelstein Lelbach

66

#ifdef NDEBUG#define assert(condition) ((void)0)#else#define assert(condition) /* … */#endif

assert.h

66Copyright (C) 2019 Bryce Adelstein Lelbach

67

namespace std { /* ... */ }#define NULL /*see definition*/#define offsetof(P, D) /*see definition*/

cstddef

67Copyright (C) 2019 Bryce Adelstein Lelbach

68

import <foo>;

Ill-formed if foo is not importable.

Your implementation defines the set of importable headers.

68Copyright (C) 2019 Bryce Adelstein Lelbach

69

“If the header identified by the header-name denotes an

importable header, the preprocessing directive is instead

replaced by the preprocessing-tokensimport header-name ; ”

[cpp.include] p7

69Copyright (C) 2019 Bryce Adelstein Lelbach

70

#include <vector>#include <iostream>

// …

Your Code

70Copyright (C) 2019 Bryce Adelstein Lelbach

71

import <vector>;import <iostream>;

// …

#include <vector>#include <iostream>

// …

Your Code Compiler Interpretation

71Copyright (C) 2019 Bryce Adelstein Lelbach

72

“If the header identified by the header-name denotes an

importable header, the preprocessing directive is instead

replaced by the preprocessing-tokensimport header-name ; ”

[cpp.include] p7

72Copyright (C) 2019 Bryce Adelstein Lelbach

73

“If the header identified by the header-name denotes an

importable header, the preprocessing directive is instead

replaced by the preprocessing-tokensimport header-name ; ”

[cpp.include] p7

“A module-import-declaration that specifies a header-name

H imports a synthesized header unit … H shall identify an

importable header.”

[module.unit] p5 s1, s4

73Copyright (C) 2019 Bryce Adelstein Lelbach

7474Copyright (C) 2019 Bryce Adelstein Lelbach

import <foo>;import <foo>;

// …import <foo>;#include <foo>

// …

Ill-formed

7575Copyright (C) 2019 Bryce Adelstein Lelbach

76

Example Extension Artifact Notes

Non-Modular Unit#include “…”…

.cpp .o

Header Unit// Created by:import <…>;

.hpp.cmi.o (optional)

Module Interface Unitexport module …;…

.ixx.cmi.o (optional)

Exactly one per module.

Module Implementation Unitmodule …;…

.mxx .o

Kinds of Translation Units

76Copyright (C) 2019 Bryce Adelstein Lelbach

77

Writing Modules77Copyright (C) 2019 Bryce Adelstein Lelbach

78

import boost.spirit;import ctre;import blas.level1;

Module names are dot-separated identifiers.

78Copyright (C) 2019 Bryce Adelstein Lelbach

79

import boost.spirit;import ctre;import blas.level1;

Module names are dot-separated identifiers.

Dots in module names have no semantic meaning.

79Copyright (C) 2019 Bryce Adelstein Lelbach

80

“A module unit is a translation unit that contains a module-declaration.”

[module.unit] p1 s1

80Copyright (C) 2019 Bryce Adelstein Lelbach

81

module a;// …

Module Implementation Unit

export module a;// …

Module Interface Unit

81Copyright (C) 2019 Bryce Adelstein Lelbach

82

module a;// …module b;// …

Only one module declaration per translation unit.

82Copyright (C) 2019 Bryce Adelstein Lelbach

83

export module a;// …export module b;// …

Only one module declaration per translation unit.

83Copyright (C) 2019 Bryce Adelstein Lelbach

84

“In a module unit, all module-import-declarations shall

precede all other top-level-declarations in the top-level-

declaration-seq of the translation-unit …”

[module.import] p1 s1

84Copyright (C) 2019 Bryce Adelstein Lelbach

85

export module …;import …;…

Module Unit Structure

85Copyright (C) 2019 Bryce Adelstein Lelbach

86

export declaration

export {declaration …

}

86Copyright (C) 2019 Bryce Adelstein Lelbach

87

export void f();

export struct A;

export int i{0};

87Copyright (C) 2019 Bryce Adelstein Lelbach

88

export {void f();struct A;int i{0};

}

88Copyright (C) 2019 Bryce Adelstein Lelbach

89

export template <typename T>T square(T t) { return t * t; }

89Copyright (C) 2019 Bryce Adelstein Lelbach

90

export template <typename T>struct is_const : false_type {};

export template <typename T>struct is_const<T const> : true_type {};

90Copyright (C) 2019 Bryce Adelstein Lelbach

91

export namespace foo { struct A; }

namespace foo { struct B; }

91Copyright (C) 2019 Bryce Adelstein Lelbach

92

export namespace foo { struct A; }

namespace foo { struct B; }

Only foo::A is exported.

92Copyright (C) 2019 Bryce Adelstein Lelbach

93

export typedef int int32_t;

export using unsigned uint32_t;

93Copyright (C) 2019 Bryce Adelstein Lelbach

94

struct A { /* … */ };

// ...

export using A;

94Copyright (C) 2019 Bryce Adelstein Lelbach

95

export import a;

95Copyright (C) 2019 Bryce Adelstein Lelbach

96

export module square;

export template <typename T>T square(T a) { return a * a; }

square.ixx

export module add;

export template <typename T>T add(T a, T b) { return a + b; }

add.ixx

export module math;

export import square;export import add;

math.ixx

96Copyright (C) 2019 Bryce Adelstein Lelbach

97

export module a;

struct S { int m; };export S foo();

a.ixx

import a;

int main() {auto s0 = foo();s0.m = 42;

}

main.cpp

97Copyright (C) 2019 Bryce Adelstein Lelbach

98

export module a;

struct S { int m; };export S foo();

a.ixx

import a;

int main() {auto s0 = foo();s0.m = 42;

S s1{};}

main.cpp

98Copyright (C) 2019 Bryce Adelstein Lelbach

99

export module a;

struct S { int m; };export S foo();

a.ixx

import a;

int main() {auto s0 = foo();s0.m = 42;

decltype(foo()) s1{};}

main.cpp

99Copyright (C) 2019 Bryce Adelstein Lelbach

100

#pragma once

namespace std {export template <typename F, typename... Args>/* unspecified */ bind(F&& f, Args&&... args);}

functional

Textual Inclusion

100Copyright (C) 2019 Bryce Adelstein Lelbach

101

#pragma once

namespace std {template <typename F, typename... Args>struct __binder;

template <typename F, typename... Args>__binder<F, Args...> bind(F&& f, Args&&... args);}

functional

Textual Inclusion

101Copyright (C) 2019 Bryce Adelstein Lelbach

102

#include <functional>

int main(){using namespace std::placeholders;

auto add_four0 = std::bind(std::plus{}, _1, 4);

std::__binder<std::plus<>, …, int> add_four1= std::bind(std::plus{}, _1, 4);

}

main.cpp

Textual Inclusion

102Copyright (C) 2019 Bryce Adelstein Lelbach

103

export module std.functional;

namespace std {export template <typename F, typename... Args>/* unspecified */ bind(F&& f, Args&&... args);}

functional.ixx

Modular Import

103Copyright (C) 2019 Bryce Adelstein Lelbach

104

export module std.functional;

namespace std {template <typename F, typename... Args>struct __binder;

export template <typename F, typename... Args>__binder<F, Args...> bind(F&& f, Args&&... args);}

functional.ixx

Modular Import

104Copyright (C) 2019 Bryce Adelstein Lelbach

105

import std.functional;

int main(){using namespace std::placeholders;

auto add_four0 = std::bind(std::plus{}, _1, 4);

std::__binder<std::plus<>, …, int> add_four1= std::bind(std::plus{}, _1, 4);

}

main.cpp

Modular Import

105Copyright (C) 2019 Bryce Adelstein Lelbach

106

Visible: In scope, can be named.

Reachable: In scope, not necessarily namable.

106Copyright (C) 2019 Bryce Adelstein Lelbach

107

export module a;

struct S { int m; };export S foo();

a.ixx

import a;

main.cpp

In main.cpp:

• S is reachable.

• foo is reachable and visible.

107Copyright (C) 2019 Bryce Adelstein Lelbach

108

Modules enable true encapsulation.

108Copyright (C) 2019 Bryce Adelstein Lelbach

109

#pragma once

// Implementation detail/not part of the API.template <typename T>struct id { using type = T; };

b.hpp

#include “a.hpp”#include “b.hpp”

main.cpp

#pragma once

// Implementation detail/not part of the API.template <typename T>T id(T t) { return t; }

a.hpp

Textual Inclusion

109Copyright (C) 2019 Bryce Adelstein Lelbach

110

#pragma once

// Implementation detail/not part of the API.template <typename T>struct id { using type = T; };

b.hpp

#include “a.hpp”#include “b.hpp”

main.cpp

#pragma once

// Implementation detail/not part of the API.template <typename T>T id(T t) { return t; }

a.hpp

Textual Inclusion

110Copyright (C) 2019 Bryce Adelstein Lelbach

111

#pragma once

namespace b::detail {template <typename T>struct id { using type = T; };}

b.hpp

#include “a.hpp”#include “b.hpp”

main.cpp

#pragma once

namespace a::detail {template <typename T>T id(T t) { return t; }}

a.hpp

Textual Inclusion

111Copyright (C) 2019 Bryce Adelstein Lelbach

112

#pragma once

namespace b::detail {template <typename T>struct __dont_use_id { using type = T; };}

b.hpp

#include “a.hpp”#include “b.hpp”

main.cpp

#pragma once

namespace a::detail {template <typename T>T __dont_use_id(T t) { return t; }}

a.hpp

Textual Inclusion

112Copyright (C) 2019 Bryce Adelstein Lelbach

113

#pragma once

namespace b::detail {template <typename T>struct __dont_use_id { using type = T; };}

b.hpp

#include “a.hpp”#include “b.hpp”

main.cpp

#pragma once

namespace a::detail {template <typename T>T __dont_use_id(T t) { return t; }}

a.hpp

Textual Inclusion

export module b;

template <typename T>struct id { using type = T; };

b.ixx

import a;import b;

main.cpp

export module a;

template <typename T>T id(T t) { return t; }

a.ixx

Modular Import

113Copyright (C) 2019 Bryce Adelstein Lelbach

114

No more detail/impl namespaces.

No more uglifying identifiers.

114Copyright (C) 2019 Bryce Adelstein Lelbach

115

“A name is said to have linkage when it might denote the same object,

reference, function, type, template, namespace or value as a name

introduced by a declaration in another scope:

— When a name has external linkage, the entity it denotes can be

referred to by names from scopes of other translation units or from

other scopes of the same translation unit.

— When a name has module linkage, the entity it denotes can be

referred to by names from other scopes of the same module unit or

from scopes of other module units of that same module.

— When a name has internal linkage, the entity it denotes can be

referred to by names from other scopes in the same translation unit.

— When a name has no linkage, the entity it denotes cannot be referred

to by names from other scopes.”

[basic.link] p4

115Copyright (C) 2019 Bryce Adelstein Lelbach

116

Example Visible From Notes

External Linkage

extern void foo();export void bar();extern int i{};export bool b{};

Other translation units.

Module Linkage

struct S;int foo();int i{};

This module.

In non-modular units,

entities with module

linkage have external

linkage.

Internal Linkage

static void foo();static int i{};bool const b{};namespace { /* … */ }

This translation unit.

No Linkage

int main() {int i{};

}This scope.

Kinds of Linkage

116Copyright (C) 2019 Bryce Adelstein Lelbach

117

Modules are sandboxed.

117Copyright (C) 2019 Bryce Adelstein Lelbach

118

#include “a.hpp”#include “b.hpp”

main.cpp

#pragma once

void bar(foo f);

b.hpp

#pragma once

struct foo { /* … */ };

a.cpp

Textual Inclusion

118Copyright (C) 2019 Bryce Adelstein Lelbach

119

#include “a.hpp”#include “b.hpp”

main.cpp

#pragma once

void bar(foo f);

b.hpp

#pragma once

struct foo { /* … */ };

a.hpp

Modular Import

import a;import b;

main.cpp

export module b;

export void bar(foo f);

b.ixx

export module a;

export struct foo { /* … */ };

a.ixx

Textual Inclusion

119Copyright (C) 2019 Bryce Adelstein Lelbach

120

#include “a.hpp”#include “b.hpp”

main.cpp

#pragma once

void bar(foo f);

b.hpp

#pragma once

struct foo { /* … */ };

a.hpp

Modular Import

import a;import b;

main.cpp

export module b;

export void bar(foo f);

b.ixx

export module a;

export struct foo { /* … */ };

a.ixx

Textual Inclusion

120Copyright (C) 2019 Bryce Adelstein Lelbach

121

Non-header-unit imports do not import macros.

121Copyright (C) 2019 Bryce Adelstein Lelbach

import foo; // No macros brought in.import <foo.hpp>; // Macros brought in.import “foo.hpp”; // Macros brought in.

122

export module a;

#define foo int

a.ixx

import a;

foo f{};

main.cpp

122Copyright (C) 2019 Bryce Adelstein Lelbach

123

export module a;

#define foo int

a.ixx

import a;

foo f{};

main.cpp

The definition of foo isn’t brought in by the import.

123Copyright (C) 2019 Bryce Adelstein Lelbach

124

#pragma once

#define foo int

a.hpp

import <a>;

foo f{};

main.cpp

124Copyright (C) 2019 Bryce Adelstein Lelbach

125

#pragma once

#define foo int

a.hpp

import <a>;

foo f{};

main.cpp

The definition of foo is brought in by the import.

125Copyright (C) 2019 Bryce Adelstein Lelbach

126

Modules never see the definition of programmatic macros defined in other modules.

Programmatic macros == #define FOO

Non-programmatic macros == -DFOO

126Copyright (C) 2019 Bryce Adelstein Lelbach

127

export module a;

export struct foo{};

a.ixx

#define foo barimport a;#undef foo

foo f{};

main.cpp

127Copyright (C) 2019 Bryce Adelstein Lelbach

128

export module a;

export struct foo{};

a.ixx

#define foo barimport a;#undef foo

foo f{};

main.cpp

The definition of foo isn’t seen by the imported module.

128Copyright (C) 2019 Bryce Adelstein Lelbach

129

export module a;

#if defined(DEBUG)// …

#else// …

#endif

a.ixx

#define DEBUGimport a;

main.cpp

129Copyright (C) 2019 Bryce Adelstein Lelbach

130

export module a;

#if defined(DEBUG)// …

#else// …

#endif

a.ixx

#define DEBUGimport a;

main.cpp

The definition of DEBUG isn’t seen by the imported module.

130Copyright (C) 2019 Bryce Adelstein Lelbach

131

#pragma once

#if defined(DEBUG)// …

#else// …

#endif

a.hpp

#define DEBUGimport a;

main.cpp

131Copyright (C) 2019 Bryce Adelstein Lelbach

132

#pragma once

#if defined(DEBUG)// …

#else// …

#endif

a.hpp

#define DEBUGimport a;

main.cpp

The definition of DEBUG isn’t seen by the imported header-unit.

132Copyright (C) 2019 Bryce Adelstein Lelbach

133

Programmatic macros are:

• Never seen by modules.

• Never exported from non-header-unit modules.

133Copyright (C) 2019 Bryce Adelstein Lelbach

134

#define _LIBCPP_NO_EXCEPTIONSimport <vector>;

134Copyright (C) 2019 Bryce Adelstein Lelbach

135

#define _LIBCPP_NO_EXCEPTIONSimport <vector>;

The definition of _LIBCPP_NO_EXCEPTIONS isn’t seen by <vector>.

135Copyright (C) 2019 Bryce Adelstein Lelbach

136

“If the header identified by the header-name denotes an

importable header, the preprocessing directive is instead

replaced by the preprocessing-tokensimport header-name ; ”

[cpp.include] p7

136Copyright (C) 2019 Bryce Adelstein Lelbach

137

#define _LIBCPP_NO_EXCEPTIONS#include <vector>

137Copyright (C) 2019 Bryce Adelstein Lelbach

138

#define _LIBCPP_NO_EXCEPTIONS#include <vector>

The definition of _LIBCPP_NO_EXCEPTIONS isn’t seen by <vector>.

138Copyright (C) 2019 Bryce Adelstein Lelbach

139

#define _LIBCPP_NO_EXCEPTIONS#include <vector>

#define NDEBUG#include <assert.h>

// For `readlink`.#define _XOPEN_SOURCE#include <unistd.h>

139Copyright (C) 2019 Bryce Adelstein Lelbach

140

How do we deal with these non-modular headers?

140Copyright (C) 2019 Bryce Adelstein Lelbach

141

Modular Import

#pragma once#define _XOPEN_SOURCE // For `readlink`.#include <unistd.h>

// …

a.hpp

Textual Inclusion

export module a;#define _XOPEN_SOURCE // For `readlink`.import <unistd.h>;

// …

a.ixx

141Copyright (C) 2019 Bryce Adelstein Lelbach

142

Modular Import

#pragma once#define _XOPEN_SOURCE // For `readlink`.#include <unistd.h>

// …

a.hpp

Textual Inclusion

export module a;#define _XOPEN_SOURCE // For `readlink`.#include <unistd.h>

// …

a.ixx

142Copyright (C) 2019 Bryce Adelstein Lelbach

143

module;#pp-directive …;export module …;import …;…

Module Unit Structure

143Copyright (C) 2019 Bryce Adelstein Lelbach

144

Modular Import

#pragma once#define _XOPEN_SOURCE // For `readlink`.#include <unistd.h>

// …

a.hpp

Textual Inclusion

module;#define _XOPEN_SOURCE // For `readlink`.#include <unistd.h>export module a;

// …

a.ixx

144Copyright (C) 2019 Bryce Adelstein Lelbach

145

module;#include <boost/circular_buffer>export module boost.circular_buffer;namespace boost {export using ::boost::circular_buffer;

}

145Copyright (C) 2019 Bryce Adelstein Lelbach

146

Modules are order independent.

146Copyright (C) 2019 Bryce Adelstein Lelbach

147

import b;import a;

import a;import b; ==

147Copyright (C) 2019 Bryce Adelstein Lelbach

148

Modules cannot have cycles.

148Copyright (C) 2019 Bryce Adelstein Lelbach

149

#pragma once#include “a.hpp”

struct X;struct Y { X* x; };

b.hpp

#pragma once#include “b.hpp”

struct Y;struct X { Y* y; };

a.hpp

Textual Inclusion

149Copyright (C) 2019 Bryce Adelstein Lelbach

150

Modular Import

#pragma once#include “a.hpp”

struct X;struct Y { X* x; };

b.hpp

#pragma once#include “b.hpp”

struct Y;struct X { Y* y; };

a.hpp

Textual Inclusion

export module b;import a;

struct X;export struct Y { X* x; };

b.ixx

export module a;import b;

struct Y;export struct X { Y* y; };

a.ixx

150Copyright (C) 2019 Bryce Adelstein Lelbach

151

Modular Import

#pragma once#include “a.hpp”

struct X;struct Y { X* x; };

b.hpp

#pragma once#include “b.hpp”

struct Y;struct X { Y* y; };

a.hpp

Textual Inclusion

export module b;import a;

struct X;export struct Y { X* x; };

b.ixx

export module a;import b;

struct Y;export struct X { Y* y; };

a.ixx

151Copyright (C) 2019 Bryce Adelstein Lelbach

152

Modular Import

#pragma once#include “a.hpp”

struct X;struct Y { X* x; };

b.hpp

#pragma once#include “b.hpp”

struct Y;struct X { Y* y; };

a.hpp

Textual Inclusion

export module b;import a;

struct X;export struct Y { X* x; };

b.ixx

export module a;import b;

struct Y;export struct X { Y* y; };

a.ixx

152Copyright (C) 2019 Bryce Adelstein Lelbach

153

Modular Import

export module b;

struct X;export struct Y { X* x; };

b.ixx

export module a;

struct Y;export struct X { Y* y; };

a.ixx

import a;import b;

main.cpp

153Copyright (C) 2019 Bryce Adelstein Lelbach

154

How do we break cycles?

154Copyright (C) 2019 Bryce Adelstein Lelbach

155

a

c

b

d

x.ixx y.ixx

155Copyright (C) 2019 Bryce Adelstein Lelbach

156

a

c

b

d

x.ixx y.ixx

z.ixx

156Copyright (C) 2019 Bryce Adelstein Lelbach

157

Modules own their declarations.

157Copyright (C) 2019 Bryce Adelstein Lelbach

158

#include “a.hpp”

static void foo() { /* … */ }

b.cpp

#include “a.hpp”

void foo() { /* … */ }

a.cpp

#pragma once

void foo();

a.hpp

Modular Import

import a;

static void foo() { /* … */ }

b.cpp

module a;

void foo() { /* … */ }

a.cpp

export module a;

export void foo();

a.ixx

Textual Inclusion

158Copyright (C) 2019 Bryce Adelstein Lelbach

159

#include “a.hpp”

static void foo() { /* … */ }

b.cpp

#include “a.hpp”

void foo() { /* … */ }

a.cpp

#pragma once

void foo();

a.hpp

Modular Import

import a;

static void foo() { /* … */ }

b.cpp

module a;

void foo() { /* … */ }

a.cpp

export module a;

export void foo();

a.ixx

Textual Inclusion

159Copyright (C) 2019 Bryce Adelstein Lelbach

160

“If a declaration would redeclare a reachable declaration attached to a

different module, the program is ill-formed. As a consequence of these

rules, all declarations of an entity are attached to the same module; the

entity is said to be attached to that module.”

[basic.link] p12

160Copyright (C) 2019 Bryce Adelstein Lelbach

161

“If multiple declarations of the same name with external linkage would

declare the same entity except that they are attached to different

modules, the program is ill-formed; no diagnostic is required.”

[basic.link] p11 s2

161Copyright (C) 2019 Bryce Adelstein Lelbach

162

“[ Note: using-declarations, typedef declarations, and alias-declarations

do not declare entities, but merely introduce synonyms. Similarly, using-

directives do not declare entities. — end note ]”

[basic.link] p11 s3, s4

162Copyright (C) 2019 Bryce Adelstein Lelbach

163

export module b;

export void foo();

b.ixx

export module a;

export void foo();

a.ixx

163Copyright (C) 2019 Bryce Adelstein Lelbach

164

export module b;

export void foo();

b.ixx

export module a;

export void foo();

a.ixx

Ill formed, no diagnostic required (IFNDR).

164Copyright (C) 2019 Bryce Adelstein Lelbach

165

Modules can be contained in one file.

165Copyright (C) 2019 Bryce Adelstein Lelbach

166

#include “a.hpp”

struct pimpl { /* … */ };

a.cpp

#pragma once

struct pimpl;

a.hpp

Modular Import

module a;

struct pimpl { /* … */ };

a.cpp

export module a;

export struct pimpl;

a.ixx

Textual Inclusion

166Copyright (C) 2019 Bryce Adelstein Lelbach

167

#include “a.hpp”

struct pimpl { /* … */ };

a.cpp

#pragma once

struct pimpl;

a.hpp

Modular Import

export module a;

export struct pimpl;

module : private;

struct pimpl { /* … */ };

a.ixx

Textual Inclusion

167Copyright (C) 2019 Bryce Adelstein Lelbach

168

module;#pp-directive …;export module …;import …;…module : private;…

Module Unit Structure

168Copyright (C) 2019 Bryce Adelstein Lelbach

169

Modules can be split across multiple files.

169Copyright (C) 2019 Bryce Adelstein Lelbach

170

export module square;

export template <typename T>T square(T a) { return a * a; }

square.ixx

export module add;

export template <typename T>T add(T a, T b) { return a + b; }

add.ixx

export module math;

export import square;export import add;

math.ixx

170Copyright (C) 2019 Bryce Adelstein Lelbach

171

export module math:square;

export template <typename T>T square(T a) { return a * a; }

square.ixx

export module math:add;

export template <typename T>T add(T a, T b) { return a + b; }

add.ixx

export module math;

export import :add;export import :square;

math.ixx

171Copyright (C) 2019 Bryce Adelstein Lelbach

172

export module math.vector;export import :dot_product;float sqrt(float x);

math_vector.ixx

172Copyright (C) 2019 Bryce Adelstein Lelbach

173

export module math.vector;export import :dot_product;float sqrt(float x);

math_vector.ixx

export module math.vector:dot_product;import :sum_of_squares;import <span>;export float vector_norm(std::span<float> x) {

return sqrt(sum_of_squares(x));}

math_vector_dot_product.ixx

173Copyright (C) 2019 Bryce Adelstein Lelbach

174

export module math.vector;export import :dot_product;float sqrt(float x);

math_vector.ixx

export module math.vector:dot_product;import :sum_of_squares;import <span>;export float vector_norm(std::span<float> x) {

return sqrt(sum_of_squares(x));}

math_vector_dot_product.ixx

module math.vector:sum_of_squares;import <span>;import <algorithm>;float sum_of_squares(std::span<float> x) {

return std::transform_reduce(x);}

math_vector_sum_of_squares.mxx

174Copyright (C) 2019 Bryce Adelstein Lelbach

175

export module math.vector;export import :dot_product;float sqrt(float x);

math_vector.ixx

export module math.vector:dot_product;import :sum_of_squares;import <span>;export float vector_norm(std::span<float> x) {

return sqrt(sum_of_squares(x));}

math_vector_dot_product.ixx

module math.vector:sum_of_squares;import <span>;import <algorithm>;float sum_of_squares(std::span<float> x) {

return std::transform_reduce(x);}

math_vector_sum_of_squares.mxx

module math.vector;float sqrt(float x) { /* ... */ }

math_vector.mxx

175Copyright (C) 2019 Bryce Adelstein Lelbach

176

Example Extension Artifact Notes

Non-Modular Unit#include “…”…

.cpp .o

Header Unit// Created by:import <…>;

.hpp.cmi.o (optional)

Module Interface Unitexport module …;…

.ixx.cmi.o (optional)

Exactly one per module.

Module Implementation Unitmodule …;…

.mxx .o

Module Partition Interface Unitexport module …:…;…

.ixx.cmi.o (optional)

Module Partition Implementation Unitmodule …:…;…

.mxx .o

Kinds of Translation Units

176Copyright (C) 2019 Bryce Adelstein Lelbach

177

Modules do not force a file layout on you.

177Copyright (C) 2019 Bryce Adelstein Lelbach

178

Ecosystem Impact178Copyright (C) 2019 Bryce Adelstein Lelbach

179

#include “math.hpp”

int square(int a) { return a * a; }

math.cpp

#pragma once

int square(int a);

math.hpp How are headers found?

• Not specified by the standard.

• In practice, all implementations assume a mapping between file names and #includes.

• The file is searched for in a set of include paths.

179Copyright (C) 2019 Bryce Adelstein Lelbach

180180Copyright (C) 2019 Bryce Adelstein Lelbach

181

export module foo;

// …

bar.ixx

How are modules found?

• Not specified by the standard.

• Unlike headers, modules are programmatically named.

• A file name <-> module name mapping is not straightforward.

• Modules have to be precompiled.

• Partitions span multiple files.

181Copyright (C) 2019 Bryce Adelstein Lelbach

182

Compiled Module Interface (CMI) Configuration

• CMIs are built with a certain set of compiler options and global macro definitions (the CMI configuration).

• Ex: -Wall, -O3, -DDEBUG

• The CMI may only be used when compiling with the same set of compiler options and global macro definitions.

• Module lookup isn’t just a matter of mapping a module name to a module interface unit (MIU) + a CMI.

• It’s mapping a module name + CMI configuration to a MIU + a CMI.

182Copyright (C) 2019 Bryce Adelstein Lelbach

183

Strategies for Module Lookup

• Assume File Name == Module Name and Search

• When importing “foo”, the compiler looks for “foo.ixx” and “foo.cmi” in a set of search directories.

• If a “foo.cmi” with the wrong CMI configuration is found, an error is produced or a new CMI is built on the fly.

183Copyright (C) 2019 Bryce Adelstein Lelbach

184

Strategies for Module Lookup

• Search.

• When importing “foo”, the compiler looks for all “.ixx” files in a set of search directories.

• When the compiler finds a match it looks for a corresponding “.cmi” file with the same prefix.

• If the found CMI has the wrong CMI configuration is found, an error is produced or a new CMI is built on the fly.

184Copyright (C) 2019 Bryce Adelstein Lelbach

185

Strategies for Module Lookup

• Explicitly Passed.

• The user specifies all of the MIU and CMI files needed for compilation of a particular TU to the compiler.

• If the specified CMI has the wrong CMI configuration is found, an error is produced.

• This is one of the approaches supported by Clang today.

185Copyright (C) 2019 Bryce Adelstein Lelbach

186

Strategies for Module Lookup

• Explicit Static Mapping.

• The user specifies a file that describes a mapping from module name + CMI configuration to MIU + CMI.

• If the specified CMI has the wrong CMI configuration is found, an error is produced.

• This is one of the approaches supported by Clang today (for header units at least).

186Copyright (C) 2019 Bryce Adelstein Lelbach

187

Strategies for Module Lookup

• Client/Server Mapper Daemon

• The compiler communicates with a daemon to either request the location of an existing CMI or register a new CMI.

• This is one of the approaches being explored by GCC today.

187Copyright (C) 2019 Bryce Adelstein Lelbach

188

a.ixx b.ixx c.ixx

0.cpp 1.cpp 2.cpp 3.cpp 4.cpp 5.cpp 6.cpp

a.cmi b.cmi c.cmi

If precompilation is implicit and builds are parallel, how do we decide who builds the CMIs?

Implicit Precompilation is Problematic

188Copyright (C) 2019 Bryce Adelstein Lelbach

189

Assuming the existence of fast dependency scanners, you can implicitly discover what CMIs need to be built.

This comes with a cost, however; you’ll need to do this scanning as an additional pass prior to compilation.

Implicit Precompilation is Problematic

189Copyright (C) 2019 Bryce Adelstein Lelbach

190

Explicit Precompilation is Ideal, But…

a.i.o: a.cmi$(CC) -c a.cmi -o a.i.o

b.i.o: b.cmi$(CC) -c b.cmi -o b.i.o

c.i.o: c.cmi$(CC) -c c.cmi -o c.i.o

lib.i.o: lib.cmi$(CC) -c lib.cmi -o lib.i.o

a.cmi: a.ixx$(CC) --precompile a.ixx -o a.cmi

b.cmi: b.ixx$(CC) --precompile b.ixx -o b.cmi

c.cmi: c.ixx$(CC) --precompile c.ixx -o c.cmi

lib.cmi: lib.ixx$(CC) --precompile lib.ixx -o lib.cmi

a.m.o: a.mxx a.cmi b.cmi c.cmi$(CC) -c a.mxx -o a.m.o

b.m.o: b.mxx b.cmi c.cmi lib.cmi$(CC) -c b.mxx -o b.m.o

exe: a.m.o b.m.o a.i.o b.i.o c.i.o lib.i.o$(CC) a.o b.o a.i.o b.i.o c.i.o lib.i.o lib.so -o exe

Explicitexe: a.o b.o

$(CC) a.o b.o lib.so -o exe

a.o: a.cpp$(CC) -c a.cpp -o a.o

b.o: b.cpp$(CC) -c b.cpp -o b.o

Implicit

190Copyright (C) 2019 Bryce Adelstein Lelbach

191

Tools can no longer rely on simple lookup mechanism (include directories and header file names) to understand

C++ projects.

Dependency scanning now requires a C++ parser, not just a C preprocessor.

191Copyright (C) 2019 Bryce Adelstein Lelbach

192

Tools that want to understand C++ code will need to interface with compilers.

192Copyright (C) 2019 Bryce Adelstein Lelbach

193

C++ Ecosystem Technical Report

(P1688)

193Copyright (C) 2019 Bryce Adelstein Lelbach

194

C++ Ecosystem Technical Report:

• Packaging Modules (P1767).• Module Mapping (P1302, P1484).• Module Naming (P1634).• Module ABI.• Module Transition Path.• Compiled Module Interface (CMI) Configuration.• Compiled Module Interface (CMI) Distribution (P1788).• Dependency Scanning (P1689).• Build Performance.• Module Granularity.

194Copyright (C) 2019 Bryce Adelstein Lelbach

195

Modules are coming:

• Modules will bring substantial algorithmic build throughput improvements.

• Modules offer true encapsulation and proper sandboxing which will eliminate many structural challenges with writing C++ at scale.

• Transitioning to modules will not be free.

195Copyright (C) 2019 Bryce Adelstein Lelbach

196

Thanks:

Hana Dusíková

Gabriel Dos Reis

Richard Smith

Michael Spencer

Gašper Ažman

196Copyright (C) 2019 Bryce Adelstein Lelbach

197Copyright (C) 2019 Bryce Adelstein Lelbach

@blelbach