Thank you.
you
Test one, two. Can anyone hear us?
Perfect, Dick. One, two, one, two.
let's see yes well according to this we should have audio out there maybe i'll make it a little
bit louder. Okay. Hi, everyone. Hopefully you can hear us. I'll wait. I'll wait until
some confirmation, then we'll do a volume check. So let's, yeah. How's it going? Hi,
simo so hi so my name is richard darst i'm one of the instructors here and with me is enrico
glirion and simo tuomisto and yeah well we're just a few of the people working here to make this
course happen so we've got about 10 minutes of icebreaker kind of time to get started um
I'll quickly note that, should we have the background, the icebreakers on the screen?
Yeah, let's start with that.
So I will switch to a screen share here.
And this is called, well, we call it HackMD, but really it's not technically HackMD.
It's HedgeDock. But in it, this is sort of our home base for the workshop.
So if you click up here, it goes above the top of the screen so you can't see,
but you can switch to an edit mode and then scroll down and you see things here.
so this is markdown sorry this is markdown if you click around you can figure out how it works
you basically do what other people are doing but this is what we use to communicate so if you open
this you'll see some icebreaker questions for example what university are you from what do you
expect from this course and maybe someone can go and do a demonstration of writing things here
yes there we go so please come and open this and if you're here add like answer these questions so
there's a poll here you add o's to vote and you can also write what you expect from the course
and what's your ideal way to spend a summer day?
And while this is going, maybe we can introduce ourselves.
And actually, we'll talk about this HackMD thing
during the proper introduction in about 10 minutes.
So don't worry if it doesn't make sense.
We'll get there.
It's our audio balance.
Should we do our standard audio test?
So order me, Simo, Enrico.
Okay.
two
three
four
five
six
oh sorry seven
we lost
counting is difficult
so is the audio balanced
who needs to be higher
and who needs to be lower
yeah as you can
as you can hear from
questions that Richard is posing
this is going to be quite interactive so we try to like involve people in the in the chat and
through other methods in the actual stream so if if there's any problems or anything like that
let us know in the hackmd and um yeah that's the best way to to let us know about this
it's a tv show yeah but the interactive ones
there's a note that Enrico
is a little bit higher
so I should be lower
I can just try not to scream
I can fix my zoom
settings hopefully
I've been a bit
unlucky with hardware
in the past 24 hours
but
yeah
anyway
Okay. Let's see. So let's introduce ourselves a little bit. So my icebreaker question to
the other instructors is, why are you here? And you can interpret that however you would like.
I can start. So why am I here? Why am I in Finland? I'm from Italy. I ended up here
in this cold country. Of course, you know, I chose to be here. I didn't. And or kind of,
I don't know if I just appeared, but I'm happy to be here.
But long story short, I was a researcher.
And then the most of my time I was enjoying helping people.
And in the end, I think I joined one of the best team that is trying to help many people at Alto in the Nordics and also anywhere else in the planet when it comes to this issue.
So scientific competence.
So I'm very happy to be here with you and with anyone else who is watching from home.
And then Simo.
That's a tough question.
act to follow yeah uh so so yeah uh i'm here of course because i'm paid to do this but
but at the same time like uh yes like you mentioned it's it's very good very nice to
help people with their problems so my background is in physics but uh throughout my studies i
realized quite quickly that uh my expertise is not necessarily knowing what the physics
how the physics works but how to solve the things and and in that computing is nowadays like quite
commonly used and throughout the years uh there's uh there's a lot of a lot of different computing
problems that people have and uh it's always nice to help people and and get that way a contribution
to the science even if you don't do the science yourself yeah and then richard where are you here
that's a good question so i guess you could say i'm here because i was a researcher
i did a postdoc at alto and i enjoyed the research but i enjoyed the scientific computing part of
things rather than the applying for grants and writing articles and papers. So I slowly tried
to move myself that way until I found a job in this team that's putting on this course. And well,
yeah, I guess the rest sort of developed naturally.
so yeah that's my story
um
yeah let's take a look let's see what people are writing here for our icebreaker so
yeah so far we see people from all to Helsinki Tampere if you're just joining now and you're
registered for the course, in your email, you should have a link to this HackMD document.
Well, not HackMD, it's notes.coderefinery.org. If you open this and switch to edit mode, which is
right up here past the screen, then you can scroll down and answer these questions. So
add a vote to this poll for where you are and write down a little bit about why you want to be
in this course so we see people one yeah yeah as we can see in the in the poll that
we have a wide audience not only from alto but other universities as well welcome
if you're probably not
familiar with our faces
maybe some other people aren't either
but it's good to have
a lot of people here from different places
and we hope that the
materials and the teaching
we are going to present is going to
be applicable in all
kinds of like HPC situations so
we try to make it so that it's
more generic than just
simply also
yeah
i wanted to add that it's important that everyone tests this notes.code refinery.org
because we can see you know if if all of you would be start editing right now do we need to increase
you know the the resources of the server but so far there's 40 people writing in there and seems
seems that it's fine i really like this what's your ideal way to spend the summer day
I feel like doing them all during the sample.
Okay, so it is now noon.
So I guess we should start with the proper intro.
So I will switch to my screen share.
Actually, Enrico, can you let me share on Zoom?
Actually, I probably can without you doing anything.
Okay, it works, and if I push here, yes, here we go.
So this is the workshop page, and well, if I move the stuff, you can see the URL, but
You probably can find that yourself in the email or in the Twitch description.
So if we scroll down a little bit, we get to the schedule.
So first off, at the top, there's some basic important information, which we'll keep updating every day.
So keep checking back.
But schedule.
Yes, so this is our home base and how our second home base and how you can find the different links where we are.
So I'm going to open the summer kickstart intro here.
And as usual, there's far more information here that I can possibly talk about.
So let's see how much we can do.
Okay, so intro, who are we?
The team that you're seeing here is called Science IT or Alto Scientific Computing.
We work in collaboration with other universities sometimes, and there's one talk in collaboration with CSC. So together we're putting on this course and trying to make sure you have everything you need. Most of the teachers are from Aalto University.
so the contents of this course so day one is sort of general big picture things so there's a little
bit of hands-on to do but for the most part it's sort of demos and talking and learning about
each other and how we do stuff and so on but then once we get to days two and three there's a lot
of hands-on and we're actually going to use the computer cluster attending the course is multiple
different people so we have people from different universities using different clusters
so even though our demos are based on ulta university we try to adapt our materials so
it's suitable to others but you might need to figure out a little bit other things if you're
at other sites so practicalities how this workshop works so as you may notice we are a live stream
so this is not zoom you can't send a chat message directly to us so i would look at it as like a tv
production which i think was in brico's metaphor so we're all here watching we're broadcasting to
the world there's um yeah and then there's pauses like commercial breaks where you can go and you
can work on the exercises yourself or in groups or however you're doing it there is a zoom session
for live help during the exercises which you can be in if you'd like but the main way we
communicate is the hackmd which i will show shortly so the workshop has many different
parts there's a lot of talking and demos there's some where we'll be typing and you're watching
there's sometimes where we will say okay now it's your time to work and we mute the stream and then
you have 10 or 15 or 20 or 30 or however many minutes to work on stuff yourself before we come
back and discuss and then there's breaks so at least the last 10 minutes of each hour or so
we would like to have our break time so to communicate so the twitch chat we don't really
watch that much so there's basically too much for us to be able to see that but you can sort of try
it if you have practical questions. But HackMD is the most important way to talk. And you can
see an example here. So we have section headings and subsection headings for where we are.
And then there'll be questions and then answers come in a sub bullet point. And the great thing
here is anyone can ask at any time and you can sort of do it live and you don't have to interrupt.
So even though this is live stream, the number of questions we have is sort of far larger than I've ever experienced in an in-person or Zoom course because it's anonymous and asynchronous.
You will get used to this as we do it.
So if you see the people that know, yeah, it'll start making sense.
the main point to remember is to ask questions at the bottom always at the top there's different
buttons to switch between view and edit mode so switch to edit mode you can write stuff and then
back to view mode so we noticed that if people are in view mode that increases performance a
bit so might have less chance of the whole system crashing um yes so where do you focus so
there's the screen sharing lecture which is the first thing you should be focusing on
when we're talking during the times that where you're typing you can be working on that hackmd
there'll be more information than you can possibly follow during the course so um look at it when you
need to but don't let it distract you and then of course there's the lesson web page to refer to
things the reason we have this really interesting vertical sharing mode is because so that you can
do this with twitch you can make part of it vertical and then have the other half of your
screen for your own work so it actually works pretty well so hide the chat go to theater mode
and then well you have plenty of space to follow along or even do your other work while you're
watching what we're doing works pretty well we try to be accessible to many different learning
styles and that's because mainly the recordings and the hackmd so if you want to just watch and
follow on later you can the recordings will be available um that'll be available tonight even
and that's possible because there's no privacy concerns here any questions you ask the writing
can be there but there's no way for your voice or video to be in the recordings uh regarding this
it's good idea to remember to not post like private information to the hack committee or
into the chat um like don't put passwords or or your email addresses or anything like that into
the into the hackmd because well the hackmd might be shown on the stream yeah yes okay um
yeah and there's different paths you can take so some people might be really active about watching
and doing all the exercises some people might prefer to just watch and then do them later
either way is okay we won't get upset and we also won't even know
please remember to be nice to each other so everyone here is at different level everyone
here is both a teacher and a learner somehow um if you're with other people talk to them and ask
them how it's going and provide the help that they might need and once something isn't going well
speak up immediately so we're always watching the hackmd so if you write in the bottom hey i can't
see the screen you're not sharing the right thing or you're too quiet right and we will see and fix
it so what can go wrong this is fun so yeah so first off you can get overloaded with information
not just you can but you will get overloaded with information so don't worry once it gets like this
then change your watching style and watch some and come back later don't get so caught up trying
to keep up that you sacrificed the rest of the course if there's too much information in hackmd
to follow so that's expected that's going to happen our typical courses get several hundred
questions per day so don't try to follow it read it later for future reference or read the answers
to your questions if hackmd lags and you can't edit or it's going wrong then that means it's
getting a bit too long and for that we will archive the other the old questions to an
archive link which is located at the top of hackmd if we don't have time to cover everything well
that actually will happen so then we summarize and we leave you things that you can review later
so a lot of these lessons are more starting points to the lessons rather than trying to do everything
with you right now if we deviate from the schedule well we're sorry we try to stay on track but this
is going to happen especially if the audience gets really interested in something we'll spend
more time there and reduce time from other things if there's major accessibility problems
like for example the audio quality is bad or the screen is not visible things are too small we're
typing too fast then write the problem in the bottom of hack and d immediately and we'll notice
and fix it if the stream suddenly dies that means that something has happened to my internet
connection and my streaming machine in that case just stick around we'll be back in about five
minutes or so it takes a while to get set up if you don't have something ready something installed
or configured ready to go well that's too bad but switch to watching mode work on it for homework
and come back the next day if you can't attend the course every day well that's okay uh we have
the videos available that's the whole reason of the live streaming so that way the videos
are available with no privacy risks if a cat visits one of us well that's not a problem that's
actually what we hope happens if the course is too cool and you want to know more about what
we're doing we have advanced information on some of the lessons ask your local supporters
and we'll search around and you can find more so final note so this is recorded and will be put on
youtube we will do a great effort to get the videos processed and on youtube by midnight tonight
so far in most courses that works you don't have to worry about being in the video because you can't
possibly be in there because you're watching a stream don't add names or personal information
in hackmd don't post personal information because it might be streamed to the world so just don't
say a name just say like yeah i'm at this university i have this problem and let us figure
out all the outputs including hackmd will get a cc by license and will be published for people
to refer to later so don't write anything private there either if you want credits for the course
we don't give certificates for this course but if you're in finland you can go to this online course
hands-on scientific computing and you can get credits and it covers a lot of the same things
we're talking about now um yeah and you can join us in helping to make courses like this possible
later so with that i'm done let's switch to hackmd and see if there's any questions yeah i'll add one
thing that that throughout like every day uh through the through every day we asked um ask
usually to give some feedback some something good something uh bad something that can be improved
upon and that sort of thing so please leave the feedback because that helps us design the courses
for the next years and maybe sometimes even the next days because if we notice that the pace is
too fast base is too slow something was not gone through in fully in in the day's materials we can
then adapt on that so please give us feedback yeah okay so if there were any questions you
could be writing them right here right now and we'll see even after we go on to the next section
so with that being said
I believe it's Enrico
next is that correct
yep I'm going to take over the
screen share
yes please do
so
alright
I'm sharing the screen
there you go and with that
I will be off for a few sections so it's to Enrico and Simo now so see you later bye thanks
so this small 10 minutes the link to the materials that you're watching right now
is um is in the schedule I will really not go through everything basically we will just go
through the pictures to kind of give the basic terms and the basic idea of where you normally
you are when you do scientific computing and where you want to be when you need to scale up your
your resources so it's always good to start with some xkcd comics if you're not familiar with them
you know you can spend a few hours checking or a few days checking them all but this is a typical
case when you ask someone you know what is your is this your machine learning pipeline system
and you know it's basically just a blob of various stuff some data comes inside something happens
some algebra happens and then you know you get something out and if it's not good you mix it a
bit so this joke here is kind of you know most of you are doing or will be doing some sort of
you know computing in the sense that you have some data you need to process the data instruct
something meaningful out of the data and then eventually if that's your task you might write
a paper about it or publish it or whatever yes so so yeah you might you might take some numbers
turn the use computers to turn the numbers to different numbers and then you publish them in
a paper and then you get more citations and that sort of thing exactly so if this is kind of the
average process the point of view that we are taking here is more like okay you know in this
process part what is actually happening and most likely the the configuration that you have or that
you've been having is that you have a laptop or a desktop computer which is a piece of hardware
and the piece of hardware inside has different parts so that's the cpus which is like you know
the central processor unit basically where the math is kind of happening where the reasoning
it's happening but of course the cpu alone doesn't really do much the cpu needs to access some data
and so there's the memory where there is the data that is can be accessed fast by the cpu and then
there's the hard disk in your machine where you might have larger larger files of course you know
having these three things will already be enough well i didn't write the screen you also need a
screen and a keyboard to interact with this hardware human interface devices exactly and
a chair maybe because you know you don't wanna stand all day but of course you might also have
a network connection which brings you to the internet so that sometimes you can pull let's
say that some of your data is stored in some google drive or cloud or whatever cloud system
dropbox then some computers not all of them might also have these gpus which the the g is stands for
graphical processing unit so they're like many many many cpus packed all together and they can
be very efficient for for for doing many kind of parallel parallel things parallel computations we
don't need to go right now into details and then on top of that you have your operating system
the software you've been using python or matlab whatever and then yes the user so this is what
what you're very familiar with and now you have to think that you want to maybe move away or maybe
you need to move away from your laptop because your hard disk is not you know big enough to keep
all the data or maybe the cloud storage is is so slow to download the data from the google drive
because it's so big and your internet connection is not fast you know or maybe the ram you really
need lots of memory because you need to keep him in the ram i don't know big matrices so you
understand that at some point if the things are scaling your laptop or your local desktop is not
good enough anymore and this is where we are with this course and this is the idea here as i said i
skip all the words now you have to understand that there is like a map this is the map where we will
be moving in the next in the next days so that right now most likely you're in some you know
remote desktop machine or laptop machine connected to the internet but then you you are not really
interested anymore in running the things in your machine in in you know basically your machine is
just a keyboard and a screen and an internet connection so that you're able to access you know
some remote computational computational resources so this of course also means that
your data you you don't really look anymore at the data that you have in your local machine
you start maybe moving the data to the big cluster where you might have access or you know have data
in other in other system that are not physically in your in your in your machine so that you can
you know run things basically run processes in in other in other clusters now of course with
great power comes great responsibility which means that most of the time you logging into one of
this cluster to the so-called login node which is like an entry point you see the drawing there that
is the the sign that you're not supposed to even stop your car there because the login node is just
an entry point then request what you need do you need 20 cpus do you need 200 cpus you you request
them you wait for the request and then you can start processing all your numbers it's important
to get this idea of you know where the map where you are in the map and we will this will be a theme
that will happen in all the in all the in all the lessons that we that we cover here in your
experience simon do you is it how can i say is it so familiar to you to immediately know where you
are in the various if you're in a single node if you're in the login node if you are in your home
is is this what you do daily basically yes yes it's completely like natural so so how i often
think of it like i i use this analogy once that like uh one if you if you go to a different place
like if you go to like a summer cabin or something you have again to the summer cabin summer cabin
and you go there and you don't if you don't see it the first time then you you figure out where's
everything like you first have to figure out where's the where's the sauna and where's where's
the lake and that sort of things you have to figure out where the stuff is but if you own a
summer cabin every time you could go there you're not checking where the sauna is and where the lake
is because you live there and that's your place and and this is the kind of thing that happens
once you start using this system like you become acclimated to the system you become
uh you you start to like feel that you are at somewhere even if you're just using like a like
a terminal that we'll be talking about later on uh if you're using just like a
like a client without any graphics you still look and get a feel of where you are and this
is very important because that will enable you to like move some of your work like part of your
workload to to a system that has something that your laptop doesn't have like enrico spoke about
but in order to make that transition easier it's usually easy to get this kind of like a grasp of
where you are where where are you placed so throughout this course it's a good idea to ask
ask us and ask yourself okay where am i now like of course you're here in your office you're here
looking or you're in your home or somewhere you're looking at the laptop screen you're here physically
but where you're running the code where where are you where's your data where is stuff coming from
where is the cloud this kind of things these if you get a grasp of these it will make your life
lot easier yeah and this is very important because some people might have been not familiar with the
system that you actually know what's going on under the hood that you actually are super sure
that that part of the disk is physically there with you with the data and some other parts are
actually in some in some remote system and there are there are devices like ipads or or smartphones
you know well where this is it feels seamless but in the background few some data is in the actual
phone some data is in the actual cloud here when at this stage it's actually important to know
where you are to know where your data are and where your process meaning you know the actual
computations are they happening in my laptop even though the data that i'm pulling is somewhere else
in the on the planet or are they happening they are close to the data so that they can be you
know as fast as and efficient as possible in general the final comments because we are only
have two minutes left for this part is that the use of remote computational resources is also
a sustainable way of doing computations for two things for yourself because you don't need to keep
your laptop open during the night to run whatever machine learning things you need to do you can
start your processes let them you know put them in the queue wait that it's they wait that it's
their turn and then the processes start they get all the job and at the end you know you check the
morning after and everything is done and you can then just look at the data and write your paper
and another thing of course is that if your process is is over you know then someone else
will use that specific computational resource so you don't you know if if you leave your laptop on
all night to do something and then it only took one hour it's a bit of a waste of energy you know
to leave the laptop on all night of course you know laptops are have some system to save energy
and and every every computer has it but the idea here that if there's a big machine that we can all
share uh how can i say um equally you know or in in a fair way then it's it's more efficient for
for for everyone and also even one can think it even for the planet it's even it's even better
i don't have any other comments i don't know simo if you want to add something we basically have one
minute left but i'll mention that there was a question in the hackmd about like how much
nowadays uh you need to know about what happens below the surface and this is a good question
and like what sort of hardware things you need to know about and i would say that we are going
to be talking about these things because they relate to what your program is doing and your
program needs to understand these things and your program is basically the thing that that will deal
with the hardware eventually so if you know a bit about how your program works and how the
what the hardware is it will help you like match them together like basically if you if you know
that you need to like i don't know you you bought a big big table from ikea or somewhere and you
need to take it home you need to know the size of the the band that you are taking like can your car
fit that table in and these sort of questions are basically what you need to be thinking about and
we'll be talking about them later so so how to match what you want to do with the resources
provided by these hpc systems and and this is something where you need to know a bit about
hardware but it's not something that you need to like constantly think about but but it's good to
get uh like a sense of scale also for these kinds of things and we'll be talking about this later
all right if there's nothing else to mention from the academy i could now switch into this