1
00:00:07,277 --> 00:00:11,417
Continuous integration. Who here uses continuous integration?

2
00:00:11,637 --> 00:00:12,857
Hopefully I'm going to see lots of hands.

3
00:00:14,377 --> 00:00:17,797
Excellent, lots of hands. I had hoped for a few more, but we'll run with it.

4
00:00:18,177 --> 00:00:22,717
Every repository should always have continuous integration of some form or another.

5
00:00:22,997 --> 00:00:27,377
It's the only way to make sure that your project builds outside of your own

6
00:00:27,377 --> 00:00:30,657
machine, because you could have all sorts of weird things on your machine and

7
00:00:30,657 --> 00:00:34,177
there are all sorts of reasons why it might build on your machine and not build

8
00:00:34,177 --> 00:00:37,357
on other people's machines. So CI helps stop this.

9
00:00:38,257 --> 00:00:43,617
It will compile, test your code on every single change that's ever made to your

10
00:00:43,617 --> 00:00:44,697
code, which is a good thing.

11
00:00:44,837 --> 00:00:49,657
It catches problems before they snowball and later on blow up in your face.

12
00:00:49,917 --> 00:00:55,997
You can catch bugs and stop all sorts of other things that you don't want to happen from happening.

13
00:00:56,737 --> 00:01:01,957
CI, the way we have it done, is always done in a clean and reproducible environment.

14
00:01:02,437 --> 00:01:05,457
Every single Linux build that's run, every single Windows build that's run,

15
00:01:05,657 --> 00:01:09,037
every single free BSD build that's run, every single Android build that's run,

16
00:01:09,177 --> 00:01:11,677
all happens in a clean ephemeral Docker container.

17
00:01:12,917 --> 00:01:16,777
There is no such thing as an environment that is dirty.

18
00:01:17,117 --> 00:01:23,197
And most importantly of all, CI should never be limited to just one kind of code.

19
00:01:23,617 --> 00:01:31,597
You can do it for not just C++, but you can lint your YAML, verify your JSON.

20
00:01:32,377 --> 00:01:35,977
You can verify all your Python to make sure that it's following best practices.

21
00:01:36,297 --> 00:01:38,177
There's lots of tools out there for this.

22
00:01:38,737 --> 00:01:42,217
It's even implemented in some of our own repositories. And in fact,

23
00:01:42,217 --> 00:01:45,057
it's used by the CI notary service service that we have

24
00:01:45,057 --> 00:01:47,937
that we'll cover off later and there are also tools to

25
00:01:47,937 --> 00:01:51,337
validate your licensing to make sure that you're complying with the necessary

26
00:01:51,337 --> 00:01:56,397
kind of standards using proper open source licensing have the necessary comments

27
00:01:56,397 --> 00:02:01,397
and tags on there to verify that this code is licensed in this fashion so that

28
00:02:01,397 --> 00:02:05,377
you always know what particular license a bit of code is using.

29
00:02:07,988 --> 00:02:15,108
Generated documentation is accepted yes setting up continuous integration in a gitlab setting,

30
00:02:15,748 --> 00:02:20,548
it's done through your gitlab ci.yml file and it's held at the top end of your

31
00:02:20,548 --> 00:02:23,948
repository for the given branch that you're currently in different branches

32
00:02:23,948 --> 00:02:26,048
can have different ci setups,

33
00:02:26,628 --> 00:02:31,328
if necessary where possible please use our templates that we've kind of already

34
00:02:31,328 --> 00:02:36,928
got set up for you no need to roll your own we already have extensive tooling in place that lets you,

35
00:02:38,288 --> 00:02:41,968
get started with essentially no cost involved.

36
00:02:42,188 --> 00:02:47,068
You just add a little include setup like we've got here on screen, and you're good to go.

37
00:02:47,288 --> 00:02:51,028
If you do have special requirements, feel free to roll your own.

38
00:02:51,248 --> 00:02:54,848
GitLab has extensive documentation on how to do this. It's outside the scope

39
00:02:54,848 --> 00:02:58,948
of this talk though, because as a general rule, we recommend using the templates

40
00:02:58,948 --> 00:03:02,268
that are currently kind of rolled out and ready for everyone to use.

41
00:03:02,548 --> 00:03:06,528
And those templates cover everything everything from FreeBSD,

42
00:03:06,628 --> 00:03:12,348
Android, Windows, as well as two different kinds of Linux, glibc and Musil.

43
00:03:12,428 --> 00:03:16,968
And there's also static builds available, I believe, for Linux and Windows.

44
00:03:17,208 --> 00:03:18,228
But don't quote me on that.

45
00:03:18,968 --> 00:03:20,748
There's quite a bit there.

46
00:03:23,121 --> 00:03:26,881
You might say okay i've included those templates but how do i get my project

47
00:03:26,881 --> 00:03:27,641
dependencies available.

48
00:03:28,621 --> 00:03:31,321
For this there's two different ways of doing

49
00:03:31,321 --> 00:03:35,121
it depending on the kind of dependency you are dealing with if

50
00:03:35,121 --> 00:03:40,281
it's a non-kde project you will need to make sure it's included within the system

51
00:03:40,281 --> 00:03:45,881
images that we have those system images those docker containers are maintained

52
00:03:45,881 --> 00:03:50,741
in a central repository that we have at sysadmin ci images it's a bunch of docker

53
00:03:50,741 --> 00:03:52,701
files if you need to add something,

54
00:03:52,941 --> 00:03:57,021
you can send a merge request there, we'll get it merged in, the image will get

55
00:03:57,021 --> 00:04:00,721
rebuilt and then that dependency will be available for all projects to use.

56
00:04:01,301 --> 00:04:06,741
Relatively clear, simple and straightforward way of getting something added in for everyone to use.

57
00:04:07,621 --> 00:04:14,861
And then there's internal dependencies. We have that specified in our kde.ci yml files.

58
00:04:15,601 --> 00:04:18,241
Those let you depend on given even

59
00:04:18,241 --> 00:04:22,521
dependencies of other KDE projects in a relatively straightforward way.

60
00:04:22,721 --> 00:04:28,461
And you can also do it on a platform by platform basis, because sometimes there's

61
00:04:28,461 --> 00:04:34,101
varying reasons why you might want to, say, pull in Network Manager on Linux and FreeBSD.

62
00:04:34,381 --> 00:04:39,881
But for your Windows builds or your Android builds, Network Manager is not there. It means nothing.

63
00:04:40,141 --> 00:04:44,001
You don't depend on it or use it there. So you can just not have it available.

64
00:04:44,601 --> 00:04:47,421
Makes for really easy ways of doing things.

65
00:04:48,221 --> 00:04:53,321
There are also ways of doing bulk rebuilds of dependencies.

66
00:04:53,741 --> 00:04:57,761
For those of you who are interested in bringing along a new platform or whenever

67
00:04:57,761 --> 00:05:02,141
we upgrade a platform that's particularly used for cute version updates.

68
00:05:03,301 --> 00:05:08,021
We have something called seed jobs that essentially go rebuild all these projects

69
00:05:08,021 --> 00:05:10,721
for a given branch at once.

70
00:05:11,381 --> 00:05:14,701
The job just methodically moves through them one by one by one by one by one

71
00:05:14,701 --> 00:05:18,681
and you get an up-to-date package registry. Thank you.

72
00:05:24,516 --> 00:05:32,736
So, besides the dependencies, there's a few other things that can be configured in the KDCI YAML file.

73
00:05:33,336 --> 00:05:38,676
The probably most important setting, or the most widely used setting,

74
00:05:38,816 --> 00:05:43,336
is making passing unit tests mandatory.

75
00:05:44,416 --> 00:05:47,976
If you would be starting from scratch nowadays, that would obviously be the default.

76
00:05:48,456 --> 00:05:52,516
But from where we are coming from historically, that wasn't an option.

77
00:05:54,816 --> 00:05:58,696
So whenever you have passing tests, reliably passing tests, you want to enable

78
00:05:58,696 --> 00:06:00,896
that. That's a setting per platform.

79
00:06:02,556 --> 00:06:07,636
Not looking at any platform in particular, but some platforms are more challenging than others.

80
00:06:07,876 --> 00:06:13,096
So you might only be able to do that for, say, Linux or something like that.

81
00:06:16,036 --> 00:06:19,756
Another common setting in there that you might need is passing custom CMake

82
00:06:19,756 --> 00:06:22,876
options to your build. So if you have...

83
00:06:24,516 --> 00:06:28,896
Say some demos or examples included in your repository that aren't usually built

84
00:06:28,896 --> 00:06:35,116
that you need to enable explicitly then it of course makes sense to enable that on the ci as well,

85
00:06:36,676 --> 00:06:42,236
most of the other settings are somewhat more exotic so if you happen to write

86
00:06:42,236 --> 00:06:48,736
a valent compositor rather than a normal app they might be relevant you find

87
00:06:48,736 --> 00:06:53,436
all the the settings and and the defaults for them in the CI Utilities repository,

88
00:06:54,016 --> 00:06:59,196
which includes the global defaults. That's the one shown on the right.

89
00:06:59,716 --> 00:07:04,416
That gives you an idea on what you can already configure in there.

90
00:07:07,076 --> 00:07:12,076
And then the same repository contains the Python scripts driving all of this.

91
00:07:12,336 --> 00:07:15,936
So I mean, this isn't hugely complicated.

92
00:07:16,196 --> 00:07:19,636
So if you need something more, this is all extendable.

93
00:07:23,512 --> 00:07:31,992
And then we get to the second big part of all of this, and that is continuous delivery.

94
00:07:32,792 --> 00:07:38,012
So for us, this means building fully functional application packages.

95
00:07:39,612 --> 00:07:44,032
For the CI, we said every repository should have this.

96
00:07:45,032 --> 00:07:48,912
For CD, every application repository should have this.

97
00:07:52,472 --> 00:07:58,992
We support pretty much the same platforms as well as for the CI,

98
00:07:59,152 --> 00:08:03,672
so we have several flavors of Linux, Windows, macOS, and Android.

99
00:08:04,032 --> 00:08:08,712
We'll see a few more details in the following slides.

100
00:08:10,712 --> 00:08:16,832
On platforms where that's necessary, we also have signing and publishing,

101
00:08:17,112 --> 00:08:23,232
the signing of the packages and publishing to repositories or software stores.

102
00:08:27,872 --> 00:08:33,852
The setup is for this slightly more complicated than for CI,

103
00:08:34,072 --> 00:08:37,792
because we not just need to know how to build something,

104
00:08:38,312 --> 00:08:44,592
we also need to know exactly which dependencies to bundle, we need to know what

105
00:08:44,592 --> 00:08:50,432
it is we are actually building there, because that package ends up being user visible.

106
00:08:52,672 --> 00:08:56,052
But the first step is basically the same as for CI.

107
00:08:57,052 --> 00:08:59,712
We have predefined templates for all the supported platforms.

108
00:09:00,192 --> 00:09:04,112
You include them into your GitLab YAML file,

109
00:09:05,612 --> 00:09:09,692
and then you see a second set of jobs showing up in your build pipeline,

110
00:09:09,952 --> 00:09:16,092
which will run after your CI pipeline pipeline passed, which then builds the packages.

111
00:09:17,072 --> 00:09:22,632
And on platforms where that's necessary, you have a third stage that is then

112
00:09:22,632 --> 00:09:24,192
signing and publishing.

113
00:09:27,010 --> 00:09:32,870
By designing and publishing, this is somewhat security-sensitive,

114
00:09:33,090 --> 00:09:38,710
because this is using KDE's official package signing key.

115
00:09:41,210 --> 00:09:49,750
And the package is then sent to a repository that immediately ends up in updates on users' devices.

116
00:09:50,670 --> 00:09:54,010
So that needs to be enabled in the CI Utilities repository.

117
00:09:56,110 --> 00:09:57,950
In addition, there's YAML files

118
00:09:57,950 --> 00:10:03,250
per platform where you select which branch should get this treatment.

119
00:10:07,690 --> 00:10:12,230
Another thing we need for building packages is application metadata.

120
00:10:12,670 --> 00:10:15,610
As I said, this is now the package is user-visible, right?

121
00:10:15,670 --> 00:10:20,350
So we need to actually describe what it is that we are producing there um carl

122
00:10:20,350 --> 00:10:23,390
touched it on on this in in his talk earlier today already,

123
00:10:24,550 --> 00:10:31,170
this is all built on top of the upstream metadata file um i think most applications

124
00:10:31,170 --> 00:10:38,130
have that already it's a big xml file where you enter a description of your

125
00:10:38,130 --> 00:10:40,910
application screenshots screenshots,

126
00:10:41,170 --> 00:10:45,110
the license information,

127
00:10:45,490 --> 00:10:48,390
where to find the source code, release notes, all of that.

128
00:10:48,990 --> 00:10:51,390
And all of that hooked up to the translation system.

129
00:10:52,150 --> 00:10:56,190
And this is kind of our canonical source of choice for that data.

130
00:10:56,290 --> 00:10:57,670
We generate websites out of that.

131
00:10:58,550 --> 00:11:02,930
Flathub is consuming this directly. And for the Google and Microsoft stores,

132
00:11:03,570 --> 00:11:06,670
we are converting that in whatever format is necessary there.

133
00:11:08,484 --> 00:11:11,284
Okay, and then we get to Craft.

134
00:11:12,584 --> 00:11:19,544
So by now you probably already have heard about Craft. So Craft is basically its own distribution.

135
00:11:19,804 --> 00:11:22,644
It supports Windows, Linux and Mac.

136
00:11:23,444 --> 00:11:30,304
Recently we got additional support for Android and we use it in our CD to provide

137
00:11:30,304 --> 00:11:38,124
the packages for those three platforms, with the exception of Snap and Flatpak,

138
00:11:38,304 --> 00:11:41,704
which are not yet craft compatible.

139
00:11:43,764 --> 00:11:49,964
So craft works by building everything from scratch, but we also have a binary cache,

140
00:11:50,164 --> 00:11:58,424
so that enables us to use it on a system like a CD because we can download the

141
00:11:58,424 --> 00:12:01,764
pre-built binaries and don't have to recompile them every single time.

142
00:12:01,764 --> 00:12:05,344
So instead of days, your job runs just for a couple of minutes.

143
00:12:07,184 --> 00:12:15,224
There is craft-users-blueprints, and blueprints are basically a recipe how to build the application.

144
00:12:15,464 --> 00:12:18,344
So you specify the location where you get your source code from,

145
00:12:19,064 --> 00:12:22,844
the dependencies, and which build system to use.

146
00:12:23,104 --> 00:12:27,084
Usually you don't have to tweak anything in there, so you just provide build

147
00:12:27,084 --> 00:12:28,564
arguments that you need.

148
00:12:29,104 --> 00:12:33,784
You can also modify the install step or things like that but usually you don't

149
00:12:33,784 --> 00:12:39,664
need to do that because we build most of the things around it.

150
00:12:39,764 --> 00:12:43,684
If you still have to change something it's probably fault in your CMake file

151
00:12:43,684 --> 00:12:47,364
so you should try to follow the standard there.

152
00:12:54,644 --> 00:12:55,284
Also,

153
00:12:57,924 --> 00:13:02,204
what you should never do is... Sorry, I totally lost my thread.

154
00:13:07,004 --> 00:13:14,784
Yeah, so another thing is what you should not do is install some files from other projects.

155
00:13:14,964 --> 00:13:19,704
So try to keep your CMakeList clean, don't install third-party tools yourself.

156
00:13:20,104 --> 00:13:21,384
Craft will take care of that.

157
00:13:23,524 --> 00:13:29,484
The blueprints you'll find on Invent. You can also have third-party repositories,

158
00:13:29,564 --> 00:13:33,644
so you can have your own collection of blueprints if you want to.

159
00:13:34,164 --> 00:13:41,924
And on the CI you can configure how how Craft works a bit deeper with the Craft.ini,

160
00:13:42,084 --> 00:13:48,024
where you can specify special versions of third-party libraries or special configurations

161
00:13:48,024 --> 00:13:49,104
of third-party libraries.

162
00:13:49,964 --> 00:13:58,144
You can do it, but you should never depend on Frameworks Master because that

163
00:13:58,144 --> 00:14:03,104
will eliminate the use of the cache and the job will just take hours.

164
00:14:05,252 --> 00:14:10,772
The last thing is the ignore list you can have also in your repository by default

165
00:14:10,772 --> 00:14:15,812
we package everything and with the ignore list you can say I don't need the

166
00:14:15,812 --> 00:14:20,632
documentation of OpenSSL in my deployment so you can strip it.

167
00:14:21,412 --> 00:14:27,312
We made the experience that it's way better to remove files than just include

168
00:14:27,312 --> 00:14:32,032
specific files because then you usually miss some plugins or whatever.

169
00:14:35,412 --> 00:14:38,252
On Linux, Craft uses the app image.

170
00:14:38,612 --> 00:14:44,152
You probably all know what it is. In difference to Flatpak or Sandbox,

171
00:14:44,452 --> 00:14:48,472
you just have everything packaged into one single file.

172
00:14:48,932 --> 00:14:54,632
And it's very easy for the developer because they basically have to change nothing.

173
00:14:54,972 --> 00:14:59,752
And it's also quite simple for the user because it's just one file you run.

174
00:14:59,932 --> 00:15:07,372
So no modifications needed, but the downside, it's just some random binary from the internet.

175
00:15:09,372 --> 00:15:14,732
Our images run on most recent systems. It's all limited by ellipse C.

176
00:15:14,812 --> 00:15:21,092
So if your system is older than our build system, we are sorry,

177
00:15:21,152 --> 00:15:23,712
but we can't support your platform.

178
00:15:28,232 --> 00:15:32,232
On Windows, we support the Windows Store, which is currently opt-in,

179
00:15:32,332 --> 00:15:38,752
so you have to be in a specific config file managed by sysadmins.

180
00:15:39,012 --> 00:15:44,672
But after an initial setup of your application in the Windows Store,

181
00:15:44,912 --> 00:15:50,452
it will also be just automatically pushed to the store, and then just the maintainer

182
00:15:50,452 --> 00:15:55,572
has to click the Publish button and you're ready to go.

183
00:15:57,272 --> 00:16:03,912
So beside the Windows door, we also support the classic Windows installer, in our case, ANSYS.

184
00:16:04,012 --> 00:16:09,252
It's just an executable you run and every Windows user will know how to use it.

185
00:16:09,792 --> 00:16:14,932
And last but not least, we have a simple archive. This is just one file,

186
00:16:15,112 --> 00:16:17,572
everything compressed into one file.

187
00:16:17,712 --> 00:16:22,552
You have to extract it, but you can also run those on systems where you don't

188
00:16:22,552 --> 00:16:26,792
have administrative access. So instead of installing it, you can just run it.

189
00:16:34,402 --> 00:16:42,742
That's me again so on android we support building apks so that's the thing you

190
00:16:42,742 --> 00:16:48,682
install on your phone and aab packages that's the thing you need to submit to the google store,

191
00:16:50,102 --> 00:16:56,642
and we have two different publishing jobs the one you definitely want is the

192
00:16:56,642 --> 00:17:00,262
one publishing to to KDE's own F-Droid repositories.

193
00:17:01,102 --> 00:17:05,542
So those are software repositories similar to what you might know from Linux

194
00:17:05,542 --> 00:17:08,902
distributions. You add those to your F-Droid client on the phone.

195
00:17:09,602 --> 00:17:14,262
And when your build pushes an update to that repository, it appears on your phone.

196
00:17:16,082 --> 00:17:20,782
In the usual configuration, that means that your stable branch ends up in the

197
00:17:20,782 --> 00:17:26,762
stable repository, as one would expect. correct, the master builds end up in

198
00:17:26,762 --> 00:17:27,602
the nightly repository.

199
00:17:27,982 --> 00:17:31,802
That's a bit of a misnomer nowadays because it's actually continuous.

200
00:17:32,842 --> 00:17:34,902
The nightly is a leftover from Jenkins times.

201
00:17:36,142 --> 00:17:39,362
And that actually gives you a very quick turnaround time.

202
00:17:40,442 --> 00:17:45,482
So from a push to the master branch to a new update on your phone is a matter of minutes.

203
00:17:47,142 --> 00:17:51,442
And that's not just useful for yourself in itinerary.

204
00:17:51,442 --> 00:17:54,462
This basically gave rise to a completely new type of contribution

205
00:17:54,462 --> 00:17:57,602
that's people running bleeding edge cd

206
00:17:57,602 --> 00:18:01,162
builds in production and that gave

207
00:18:01,162 --> 00:18:09,862
us a huge boost in qa coverage because you get bug reports that are basically

208
00:18:09,862 --> 00:18:15,402
really the latest state you make a change and you get immediate feedback that

209
00:18:15,402 --> 00:18:20,642
you improved it or you fixed it right so i i certainly don't want to miss that anymore.

210
00:18:22,602 --> 00:18:26,742
The other publishing job we have is for the Google Play Store.

211
00:18:27,962 --> 00:18:32,482
That is at best semi-automatic, I would say.

212
00:18:32,642 --> 00:18:37,382
Let's emphasize on semi, not on automatic, which isn't really our fault,

213
00:18:37,562 --> 00:18:44,382
but that is due to all the paperwork and manual procedures you need to do in the Play Store.

214
00:18:44,522 --> 00:18:48,782
So it supports your publishing to the Play Store, but if you want to do that,

215
00:18:48,842 --> 00:18:51,202
that is still involving quite some manual work.

216
00:18:53,388 --> 00:18:55,888
And then we get to macOS, right? Yeah.

217
00:18:56,068 --> 00:19:01,568
So on macOS, we have two different architectures, which we also separate.

218
00:19:01,708 --> 00:19:03,988
So there are two different binary caches.

219
00:19:04,468 --> 00:19:09,108
One is the older Intel architecture, and the other one is ARM.

220
00:19:09,768 --> 00:19:13,708
What we do not have yet is the universal FET binary.

221
00:19:14,228 --> 00:19:17,828
And I think we don't plan to support it yet.

222
00:19:19,028 --> 00:19:27,168
What we get on macOS is DMG bundles, which you can just download and then by

223
00:19:27,168 --> 00:19:28,548
drag and drop easily install.

224
00:19:29,608 --> 00:19:35,808
To get that, you need to have a plist file in your application repository.

225
00:19:36,308 --> 00:19:42,968
You can either use a custom one or you can use some CMake magic to generate

226
00:19:42,968 --> 00:19:46,228
the built-in CMake plist file.

227
00:19:46,228 --> 00:19:51,308
And this is important because it contains the information about the app identity,

228
00:19:51,548 --> 00:19:59,568
which is then used by the signing service to verify it's the correct application which got signed.

229
00:20:01,888 --> 00:20:05,188
Mac OS is a bit special because it's Apple.

230
00:20:05,288 --> 00:20:10,768
And for Apple, you need actual Apple hardware to build and test things.

231
00:20:10,768 --> 00:20:16,988
So that's kind of a limitation, which also hits us on our official server systems.

232
00:20:17,488 --> 00:20:26,248
Because right now we only have one machine, one Apple machine to run everything for macOS.

233
00:20:26,888 --> 00:20:35,608
So you might experience some longer CI queue times when you're on this.

234
00:20:35,608 --> 00:20:42,628
And we are cross-compiling all Intel binaries on an ARM machine.

235
00:20:44,248 --> 00:20:51,848
Another special thing about macOS is the notarization. You can see this nice message over there.

236
00:20:53,408 --> 00:20:58,068
This message appears even if your binary is signed, but not notarized.

237
00:20:58,248 --> 00:21:01,708
Notarization means that in addition to signing,

238
00:21:01,708 --> 00:21:11,248
your binary or your app bundle is sent via an API to Apple servers and they

239
00:21:11,248 --> 00:21:18,048
run some checks and then you will get a certificate which you can include in your bundle.

240
00:21:18,188 --> 00:21:21,248
And only if you did this, this message will appear.

241
00:21:21,448 --> 00:21:26,648
There is a possibility to work it around, but it's not very user-friendly.

242
00:21:27,966 --> 00:21:33,986
Okay, so then we are leaving Craft now and coming to Flatpak.

243
00:21:35,126 --> 00:21:43,646
Flatpak is probably the most or the best imported format of all formats we presented so far.

244
00:21:44,406 --> 00:21:51,226
And there's also a nice KDE team, very active, which will help you in case of questions.

245
00:21:52,586 --> 00:21:59,486
To get Flatpak working, you have to add a Flatpak manifest.json file.

246
00:22:00,046 --> 00:22:06,086
You can also add a YAML file, but JSON is used in most KDE apps.

247
00:22:06,906 --> 00:22:16,146
And one important thing about this is that this manifest is only for nightly Flatpak builds.

248
00:22:16,146 --> 00:22:23,786
For the stable Flatpak versions, they kind of duplicate in a Git repository

249
00:22:23,786 --> 00:22:26,686
in the Flathub namespace.

250
00:22:26,966 --> 00:22:34,646
So that's not that perfect, but I think in the future we might be able to get

251
00:22:34,646 --> 00:22:36,226
rid of this duplication.

252
00:22:38,866 --> 00:22:42,226
For Flatpaks, as I said, there's this JSON manifest. manifest,

253
00:22:42,286 --> 00:22:47,546
Flatpak has a runtime which includes all Qt and KDE frameworks.

254
00:22:47,926 --> 00:22:53,506
So at least this part is shared between apps and in your Flatpak manifest,

255
00:22:53,706 --> 00:22:57,846
you only have to add specific dependencies for your app.

256
00:23:00,083 --> 00:23:08,463
One nice thing about Flatpak Nightly is that you can just download the Flatpak

257
00:23:08,463 --> 00:23:10,903
ref file, which is published on our CDN,

258
00:23:11,103 --> 00:23:17,463
and then can install it via your software center, like, for example, Discover,

259
00:23:17,743 --> 00:23:27,783
and then you will also get automatic updates on every comment that gets built from your repository.

260
00:23:29,123 --> 00:23:36,163
Yeah, so I guess that's about Flatpaks. We also have Snaps, which is another huge topic.

261
00:23:37,183 --> 00:23:43,783
We won't go into detail there. I just recommend to join Kevin's talk later,

262
00:23:43,883 --> 00:23:46,463
I think after the coffee break in this room.

263
00:23:50,823 --> 00:23:54,063
Thank you. And now finally, a couple of other things.

264
00:23:54,063 --> 00:23:57,003
The cd system is not just capable

265
00:23:57,003 --> 00:24:00,443
of building your application binaries the cd

266
00:24:00,443 --> 00:24:03,303
system is quite generic and it can do anything that

267
00:24:03,303 --> 00:24:09,323
essentially requires privilege on the ci workers or the cd workers among those

268
00:24:09,323 --> 00:24:15,063
also includes publishing packages to the python package registry pi pi and it

269
00:24:15,063 --> 00:24:20,823
is also what drives the deployment of all the static websites that kd has on our infrastructure.

270
00:24:21,083 --> 00:24:25,683
Are you seeing a Hugo site, a Jekyll site, Sphinx site, any of those sites,

271
00:24:25,743 --> 00:24:29,463
they're all published using the same continuous deployment infrastructure.

272
00:24:30,163 --> 00:24:34,723
And on the continuous integration side, we also have a bit of things for energy

273
00:24:34,723 --> 00:24:35,783
consumption measurement that's

274
00:24:35,783 --> 00:24:39,603
been mentioned in some of the talks earlier this weekend. And there's a.

275
00:24:47,512 --> 00:24:51,892
That being said, with all the things we've talked about, we're still not complete.

276
00:24:52,072 --> 00:24:55,412
There are still items that are on our wish list. There are still things we were looking to improve.

277
00:24:55,712 --> 00:24:57,712
Still things that we can do better at.

278
00:24:58,112 --> 00:25:04,292
At the moment, we don't have GUI tests on Windows because Windows Docker images

279
00:25:04,292 --> 00:25:08,492
don't support any of the Windows GUI stack. It's completely missing.

280
00:25:08,752 --> 00:25:13,032
The libraries are there, but that's all that's there. the actual components

281
00:25:13,032 --> 00:25:17,652
in Windows that those libraries leverage are completely broken and don't work.

282
00:25:18,392 --> 00:25:22,132
We'd also like to be able to run unit tests on Android. At the moment Android's

283
00:25:22,132 --> 00:25:25,952
completely cross-compiled and you can't really, not without awful emulation

284
00:25:25,952 --> 00:25:29,652
anyway, run unit tests on Android and we would need to spin up a whole Android

285
00:25:29,652 --> 00:25:31,632
stack as well so there's a bit of work to be done there.

286
00:25:32,872 --> 00:25:35,672
Automated testing of continuous deployment packages would be

287
00:25:35,672 --> 00:25:38,832
nice as well so we can validate that we actually have binaries that

288
00:25:38,832 --> 00:25:41,952
can be run rather than just build successfully someone

289
00:25:41,952 --> 00:25:44,992
downloads it and finds oh you're missing this library your

290
00:25:44,992 --> 00:25:48,652
builders are some other all the way corrupt we'd

291
00:25:48,652 --> 00:25:52,232
like to get some kind of tidy up on that at the moment those

292
00:25:52,232 --> 00:25:55,072
of you who are familiar and trying to look at things on a kind of broader

293
00:25:55,072 --> 00:25:57,952
kd level view or a release unit view

294
00:25:57,952 --> 00:26:00,872
will know that it's essentially impossible not without

295
00:26:00,872 --> 00:26:03,572
some awful scripting anyway to track where we're

296
00:26:03,572 --> 00:26:06,552
at in terms of the status so we'd like to get a bit of a status overview

297
00:26:06,552 --> 00:26:10,172
a bit of a dashboard in terms of where we're at and we'd

298
00:26:10,172 --> 00:26:13,692
also like to look at publishing to more places in particular

299
00:26:13,692 --> 00:26:17,792
as julius kind of mentioned just earlier flathub kind

300
00:26:17,792 --> 00:26:22,352
of stands out massively there because at the moment we build flatpaks ourselves

301
00:26:22,352 --> 00:26:27,912
but then have to go update it in a separate place to run through flathub's infrastructure

302
00:26:27,912 --> 00:26:32,152
to get published on flathub.org which doesn't make much sense in a world where

303
00:26:32,152 --> 00:26:34,392
for the Windows Store, we can publish ourselves.

304
00:26:34,692 --> 00:26:39,652
For the Play Store, we can publish ourselves. But on Linux, we can't publish

305
00:26:39,652 --> 00:26:41,712
ourselves. So we should probably look at solving that.

306
00:26:43,432 --> 00:26:48,872
So kind of in conclusion, since we've moved to GitLab away from Jenkins,

307
00:26:49,172 --> 00:26:53,312
things have got a lot more accessible for everyone. You can now have custom jobs.

308
00:26:53,592 --> 00:26:55,752
It's a lot easier to register jobs yourself.

309
00:26:56,592 --> 00:27:01,572
And life in general has gotten a lot better, easier and nicer and more integrated

310
00:27:01,572 --> 00:27:07,392
rather than being on a separate system on a separate site that you only have so much control over.

311
00:27:08,212 --> 00:27:11,072
Things for the most part can be done by a merge

312
00:27:11,072 --> 00:27:14,332
request or directly yourselves in your own repository including in your own

313
00:27:14,332 --> 00:27:18,632
forks you won't get any of the signing automatic uploads or anything like that

314
00:27:18,632 --> 00:27:22,272
whilst you're in your own forks that only works on the mainline repositories

315
00:27:22,272 --> 00:27:28,732
it's been enabled for but for the most part you're free to do whatever you'd like in your own forks,

316
00:27:29,412 --> 00:27:32,932
and for those of you who have additional questions would like to speak to us

317
00:27:32,932 --> 00:27:38,372
more in depth on something we have a bof session on tuesday at three o'clock

318
00:27:38,372 --> 00:27:40,532
in the afternoon known in room three, I believe.

319
00:27:41,612 --> 00:27:44,052
And with that, I'll throw the room open to questions.

320
00:27:57,917 --> 00:28:05,517
Thanks for the presentation. Question about how you are managing changes to the Docker images.

321
00:28:05,737 --> 00:28:10,197
You said that you have a special place for the declarations of what's in those images.

322
00:28:10,757 --> 00:28:15,437
When the Docker image changes, do you test everything that it actually didn't

323
00:28:15,437 --> 00:28:18,897
break stuff when you upgraded, let's say, to the latest Qt version?

324
00:28:19,737 --> 00:28:22,877
So with an upgrade to a cute version we

325
00:28:22,877 --> 00:28:25,577
normally version that one a lot more strongly that's not

326
00:28:25,577 --> 00:28:29,357
something that we just jump into the linux docker

327
00:28:29,357 --> 00:28:32,377
images are based on openSUSE tumbleweed and

328
00:28:32,377 --> 00:28:35,497
we use a specific pinned repository that targets a

329
00:28:35,497 --> 00:28:39,097
specific cute major minor version so

330
00:28:39,097 --> 00:28:42,017
there can't be just a silent upgrade underneath our feet and suddenly

331
00:28:42,017 --> 00:28:45,377
you've moved to 6.7 from 6.6

332
00:28:45,377 --> 00:28:48,597
we keep that one more tightly controlled but for

333
00:28:48,597 --> 00:28:51,417
other changes there's only so much testing we can do

334
00:28:51,417 --> 00:28:54,397
generally someone's adding a library here adding a

335
00:28:54,397 --> 00:28:57,797
library there whatever the case may be adding this

336
00:28:57,797 --> 00:29:00,877
tool that they need for their tool chain there we generally just merge those

337
00:29:00,877 --> 00:29:04,777
build them and then if it's been a while since there's been a build we'll trigger

338
00:29:04,777 --> 00:29:08,817
a seed job the nature of tumbleweed and some of the other distributions that

339
00:29:08,817 --> 00:29:13,417
are in play being rolling distributions means there's likely to be binary compatibility

340
00:29:13,417 --> 00:29:16,397
breaks so yeah to answer the question there's not really too much in the way

341
00:29:16,397 --> 00:29:19,657
of checks we do but we do our best to minimize breakage where possible.

342
00:29:20,597 --> 00:29:26,477
Yeah but i think there is um we we have the mechanisms to do uh compile time

343
00:29:26,477 --> 00:29:32,957
testing where the seat drops indirectly right what we are lacking uh and that is indeed in a problem.

344
00:29:34,137 --> 00:29:38,437
Um running unit tests on all the frameworks against an upgraded qt for example

345
00:29:38,437 --> 00:29:43,977
right so this is indirectly done via the albert job so albert triggers once

346
00:29:43,977 --> 00:29:47,537
a week a rebuild of everything thing and then we see the unit test errors.

347
00:29:47,637 --> 00:29:55,497
So, but we don't have a way to actually see iGate changes to things lower in the stack.

348
00:29:55,617 --> 00:29:59,437
And not just for the CI images, right? We have a similar problem with craft here.

349
00:30:02,121 --> 00:30:09,221
I noticed there was an enable load-sensitive option in the CI config.

350
00:30:10,261 --> 00:30:17,581
How do you deal with tests that are flaky because of various CI issues such as load?

351
00:30:23,081 --> 00:30:28,501
Flaky tests are a bit of a tricky one. Generally, we put that in the hands of

352
00:30:28,501 --> 00:30:30,041
the developer to deal with.

353
00:30:30,781 --> 00:30:35,241
I'm hoping that it's not a case of the ci system itself being flaky there is

354
00:30:35,241 --> 00:30:38,921
a parameter in one of the those that you would have seen the configuration file

355
00:30:38,921 --> 00:30:40,281
there called load sensitive,

356
00:30:41,081 --> 00:30:46,301
that was introduced for certain projects whose tests were very latency sensitive

357
00:30:46,301 --> 00:30:50,541
and when the ci nodes were busy those tests would end up failing nine times

358
00:30:50,541 --> 00:30:54,501
out of ten because the round trips they were trying to do between multiple processes

359
00:30:54,501 --> 00:30:57,701
just weren't completing fast enough because the The processor was busy compiling

360
00:30:57,701 --> 00:30:58,421
someone else's project.

361
00:30:59,641 --> 00:31:04,941
Yeah. I mean, unlike Qt, I don't think we have a special support for flaky tests.

362
00:31:05,181 --> 00:31:09,341
Uh, and I would say most of the tests that are flaky, and I mean,

363
00:31:09,361 --> 00:31:10,461
we definitely have that problem.

364
00:31:10,821 --> 00:31:15,861
Um, those are flaky everywhere, right? So it's, it's not really specific to the CR.

365
00:31:17,181 --> 00:31:24,021
Uh, do you have the ability to test with, uh, devices connected like cameras

366
00:31:24,021 --> 00:31:25,441
or microphones and things?

367
00:31:27,001 --> 00:31:31,061
Not at this time. At the moment, everything's in Docker images and Docker makes

368
00:31:31,061 --> 00:31:34,361
making devices available somewhat not straightforward.

369
00:31:34,681 --> 00:31:41,281
Also, all the servers doing these builds are sitting in a Hetzner data center in Finland somewhere.

370
00:31:41,801 --> 00:31:44,621
We can't just go attach random hardware to these things.

371
00:31:44,721 --> 00:31:48,701
We'll be moving towards VM-based builds at some point, at which point we can

372
00:31:48,701 --> 00:31:52,181
probably tell the VM, please attach this random piece of hardware.

373
00:31:52,181 --> 00:31:55,181
And so at least you might have a chance of doing something but you won't be

374
00:31:55,181 --> 00:31:59,441
able to probably say I want this particular kind of data feed to come through

375
00:31:59,441 --> 00:32:00,881
this particular hardware device.

376
00:32:04,520 --> 00:32:13,420
I have a question about examples of successful configured projects with unit

377
00:32:13,420 --> 00:32:16,520
tests, functional tests, Selenium tests, and so on.

378
00:32:16,580 --> 00:32:23,480
So you have a list of favorite projects that we can take as an example to build

379
00:32:23,480 --> 00:32:29,460
our own configurations or just get the random project.

380
00:32:30,940 --> 00:32:36,940
I think we don't have a specific list of best practice examples.

381
00:32:37,360 --> 00:32:42,040
In general, I would say the new IDE application is, so basically everything

382
00:32:42,040 --> 00:32:46,620
that Carl started is basically following the current best practice.

383
00:32:46,780 --> 00:32:54,980
So they have passing tests, they have Appium tests and have all the CI set up in a way it should be.

384
00:32:56,060 --> 00:33:01,840
The older it gets, the more or it deviates from best practice, I would say.

385
00:33:02,580 --> 00:33:07,500
But maybe it might make sense to have a few showcases that actually use the

386
00:33:07,500 --> 00:33:12,780
full thing, like all platforms, all types of tests that we support. Yeah.

387
00:33:14,760 --> 00:33:24,600
So up here, KDE Connect, we have native Android and iOS apps.

388
00:33:24,840 --> 00:33:29,080
I know we are weird in this. but how much

389
00:33:29,080 --> 00:33:33,100
could we use the the cd that

390
00:33:33,100 --> 00:33:36,140
we already have to maybe sign and

391
00:33:36,140 --> 00:33:41,140
notarize the ios app which i mean shouldn't be so different from the mac bills

392
00:33:41,140 --> 00:33:48,560
right um i think the signing and notarization is not specific to how we built

393
00:33:48,560 --> 00:33:54,020
right so feeding your apk however it is built into to that system should be the same, right?

394
00:33:55,440 --> 00:33:59,980
We indeed, you are special in that sense, and we have no infrastructure for that yet.

395
00:34:00,140 --> 00:34:03,840
But I think at least the later stages should be applicable in the same way.

396
00:34:03,940 --> 00:34:06,820
It should be relatively straightforward to tie that in. It will be you have

397
00:34:06,820 --> 00:34:11,500
to do a bit of extra custom stuff to say, please invoke the notary service to sign my application.

398
00:34:11,760 --> 00:34:14,800
But that shouldn't be a problem. The notary service doesn't care how it was

399
00:34:14,800 --> 00:34:16,880
built. It just cares that you're giving it an APK.

400
00:34:18,100 --> 00:34:23,920
And the same with publishing to Google Play. Maybe we can sit together and solve it. Thanks.

401
00:34:38,000 --> 00:34:44,460
Hi, I would like to know if there's any observability taking into account into

402
00:34:44,460 --> 00:34:51,300
the CICD as in statistics statistics or metrics on which pipelines are failing, how long they take,

403
00:34:52,000 --> 00:34:54,880
how often they fail, all this kind of thing.

404
00:34:57,027 --> 00:35:01,487
Data is probably recorded within gitlab itself but at the moment we don't really

405
00:35:01,487 --> 00:35:04,807
have a way of surfacing that information you could probably pull it out using

406
00:35:04,807 --> 00:35:10,127
graphql but that may or may not be too helpful because we've had some experience

407
00:35:10,127 --> 00:35:13,567
working with graphql in the past with gitlab and the results haven't always

408
00:35:13,567 --> 00:35:15,667
been too perfect it's needed some cleaning up,

409
00:35:16,227 --> 00:35:19,667
the visualizations you get out aren't the best but that's something we could

410
00:35:19,667 --> 00:35:22,787
look at the data will be there because gitlab keeps everything for eternity

411
00:35:22,787 --> 00:35:24,587
unless the project is deleted.

412
00:35:27,107 --> 00:35:31,047
Running tests and other CI jobs require a lot of resources.

413
00:35:31,367 --> 00:35:36,487
So now it uses the infrastructure, right?

414
00:35:36,567 --> 00:35:43,527
So maybe you can provide the ability to contribute our local resources to run GitLab on my server,

415
00:35:43,627 --> 00:35:49,247
for example, and give the ability to decrease the usage of the cloud services

416
00:35:49,247 --> 00:35:52,807
by my resources that I just have but don't use.

417
00:35:53,867 --> 00:35:58,407
So you're talking about running your own GitLab instance or? Or GitLab runner.

418
00:35:58,587 --> 00:36:03,727
So your GitLab instance will use my GitLab runner on my server.

419
00:36:03,907 --> 00:36:06,967
So I contribute my resources to setup.

420
00:36:07,427 --> 00:36:10,187
This is something that you can fully do within your own forks,

421
00:36:10,187 --> 00:36:13,167
within your own personal repositories. You can just go in there.

422
00:36:13,647 --> 00:36:18,047
It has an option to register a runner. It will tell you what you need to do to set that up and do it.

423
00:36:18,047 --> 00:36:22,107
But within the mainline repositories, at the moment, we haven't really made

424
00:36:22,107 --> 00:36:24,687
a decision on that, but chances are we'll probably want to keep that to,

425
00:36:25,307 --> 00:36:29,647
for just reliability purposes alone, keep that to just the general ones that

426
00:36:29,647 --> 00:36:32,527
a sysadmin community maintained, those particular ones.

427
00:36:32,667 --> 00:36:35,967
But for your own personal repositories, you can definitely feel free to use

428
00:36:35,967 --> 00:36:39,427
your own ones as just a case of attaching the runner and making sure that it

429
00:36:39,427 --> 00:36:42,907
has the necessary environment that matches what our jobs expect,

430
00:36:43,127 --> 00:36:45,527
which is a rootless podmin setup.

431
00:36:49,294 --> 00:36:55,454
So possibly a very stupid question, but we have Craft, we have KDE Builder,

432
00:36:55,454 --> 00:36:56,734
we have KDE Source Build.

433
00:36:57,674 --> 00:37:01,934
Can Craft do all these things or maybe KDE Builder or something else?

434
00:37:02,774 --> 00:37:07,894
Do we really need three ways of building the whole thing automatically?

435
00:37:12,074 --> 00:37:16,914
So why Craft? So why we have KDE SRC?

436
00:37:19,334 --> 00:37:22,374
Craft tries to be something different

437
00:37:22,374 --> 00:37:27,034
in a way so it concentrates more

438
00:37:27,034 --> 00:37:29,994
on creating a deployment

439
00:37:29,994 --> 00:37:36,634
in the end so you have an installer that can be distributed and it also has

440
00:37:36,634 --> 00:37:47,174
a focus on reusing this cache so it's not for everyday development or not necessarily for that,

441
00:37:47,274 --> 00:37:51,094
but for creating those installers on all the platforms.

442
00:37:53,434 --> 00:37:59,174
There is a valid point in there, though. It might be worth investigating if

443
00:37:59,174 --> 00:38:03,674
we can generate the Flatpak manifest and the Snap manifest out of the Craft

444
00:38:03,674 --> 00:38:08,114
manifest because that is duplication for the same purpose, essentially.

445
00:38:09,274 --> 00:38:12,554
But I agree, the KDE source build has a different use case.

446
00:38:15,074 --> 00:38:18,494
I understand we only have two minutes left, any final questions, last lucky last?

447
00:38:22,994 --> 00:38:25,754
Seems we've answered all the questions. Thank you everyone.