Reinforcement Learning with Unity [ML Agents 1.0]

Unity ML Agents 1.0

Have you ever wondered what Reinforcement Learning is all about? I have just produced a tutorial video to show you how to get started with Unity and Tensorflow. You might not be a game developer, and your day to day usecase might be different, but I think it is great to get started with this very visual and playful way of learning what Reinforcement Learning is.

There are fantastic papers and research projects on Reinforcement Learning, if you are interested, you should definitively check them out:

AlphaStar by DeepMind
Capture the Flag: the emergence of complex cooperative agents
Emergent Tool Use from Multi-Agent Interaction

Day 3 Notes on “Learn Linux in 5 Days and Level Up Your Career” by Jason Cannon on Udemy

Day 3

1. File and directory permissions:

When ls -l typed in the console, something like this will show up on your screen: -rw-r--r-- 1 Philipp users 10400 Sep 27 08:52 sales.data

The first character or symbol could be one of the following (Symbol – Type):
- – Regular file.
d – Directory.
l – Symbolic link.

Other characters in the permission string can be (Symbol – Permission):
r – Read
w – Write
x – Execute

Permission is different for files and directories. Read permission for example for a directory, enables you to see files in that directory.

Permission Categories:

Symbol – Category
u – User
g – Group
o – Other
a – All

Every user belongs to at least one group but can belong to many groups.
groups – Displays a user’s groups.
id -Gn – (As previous)

Secret Decoder Ring:

To go back to the example above: -rw-r--r-- 1 Philipp users 10400 Sep 27 08:52 sales.data
- – Type
rw- – User
r-- – Group
r-- – Other
If specific permission is not granted, a hyphen - will take its place.

Changing Permissions (symbolic mode):

Item – Meaning
chmod – Change mode command.
ugoa – User category: user, group, other all.
+-= – Add, subtract, or set permissions.
rwx – Read, Write, Execute.

So for example: chmod g+w sales.data
This should change:
-rw-r--r-- 1 Philipp users 10400 Sep 27 08:52 sales.data
Into this:
-rw-rw-r-- 1 Philipp users 10400 Sep 27 08:52 sales.data
To add and subtract from multiple groups:
chmod u+rwx,g-x sales.data
-rwxrw-r-- 1 Philipp users 10400 Sep 27 08:52 sales.data
Change all:
chmod a=r sales.data
Will result in:
-r--r--r-- 1 Philipp users 10400 Sep 27 08:52 sales.data

Octal mode permission change examples:

chmod 700 – “-rwx——”
chmod 755 – “-rwxr-xr-x”
chmod 664 – “-rw-rw-r–”
chmod 660 – “-rw-rw—-”
chmod 644 – “-rw-r–r–”

Change group of file:

chgrp sales sales.data

File Creation Mask:

If no mask was used permissions would be:
777 for directories
666 for files

The umask Command:

umask [-S] [mode]
umask – Shows the mask used, i.e.: 0022
umask – Shows the mask used in symbolic, i.e.: u=rwx,g=rx,o=rx

2. View Files and the Nano Editor:

The cat Command:

cat file – Display the contents of file.
more file – Browse through a text file.
less file – More features than more.
head file – Output the beginning (or top) portion of file (10 lines by default).
tail file – Output the ending (or bottom) portion of file (10 lines by default).

Navigating a file:

space bar – Advance to the next page.
enter key – Advance one line.
q key – Quit viewing the file.

-n – Number of lines.
tail -15 file.txt – Output the last 15 lines at the ending (or bottom) portion of file.
tail -f file – Display data as it is being written to the file.
To exit the real-time display hit ctrl+c

The Nano Editor:

nano – Open Nano Editor.
pico – Some Destributions use pico for the Nano Editor.
nano file.txt – Open file.txt with the Nano Editor.
Use ctrl+x to exit the file.
Use ctrl+o to save the file.
Use ctrl+g to get help.

The Vi Editor:

vi file.txt – Open file with Vi Editor.
vim file.txt – m stands for improved, more features.
view file.txt Open file with Vi Editor in read only mode.

There are three modes in the Vi Editor, you are always in one of three modes, by default when opening Vi Editor you are in command-mode.

All three modes:
Mode – Key-Binding
Command-Mode – Esc
Insert-Mode – i or a A
Line-Mode – By beginning command with a :.

Command-Mode Key-Bindings:

k – Up one line.
j – Down one line.
h – Left one character.
l – Right one character.
w – Right one word.
b – Left one word.
^ – Go to the beginning of the line.
$ – Go to the end of the line.

x – Delete character.
dw – Delete a word.
dd – Delete a line.
D – Delete from the current position.
r – Replace the current character.
cw – Change the current word.
cc – Change the current line.
c$ – Change the text from the current position.
c – Same as c$.
~ – Reverse the case of a character.
yy – Yank (copy) the current line.
y<position> – Yank the <position>.
p – Paste the most recent deleted or yanked text.
u – Undo.
Ctrl-R – Redo.
/<pattern> – Start a forward search.
?<pattern> – Start a reverse search.
When searching use lower case n to go to the next result, and upper case N to the previous result.

Insert-Mode Key-Bindings:

i – Insert at the cursor position.
I – Insert at the beginning of the line.
a – Append after the cursor position.
A – Append at the end of the line.

Line-Mode Key-Bindings:

:w – Writes (saves) the file.
:w! – Forces the file to be saved.
:q – Quit.
:q! – Quit without saving changes.
:wq! – Write and quit.
😡 – Same as :wq.
:n – Positions the cursor at line n.
:$ – Positions the cursor on the last line.
:set nu – Turn on line numbering.
:set nonu – Turn off line numbering.
:help [subcommand] – Get help.

Repeating Commands:</h4

k5 – Move up a line five times.
80i – Insert 80 times.
80i_ – Insert 80 “_” characters.

vimtutor – Tutorial for Vi Editor.

A Vim Cheat Sheet can be found here: https://www.linuxtrainingacademy.com/vim-cheat-sheet/.

The Emacs Editor:

emacs file.txt – Edit file.txt.
C-<char> – Ctrl while pressing <char>.
M-<char> – “Meta” key (alt key) while pressing <char>. Esc key can also be used as the “Meta” key.

For example:
C-h – Help.
C-x, C-c – Exit.
C-x, C-s – Save the file.
C-h t – Built-in tutorial.
C-h k <key> – Describe key.

Emacs File Navigation:

C-p – Previous line.
C-n – Next line.
C-b – Backward one character.
C-f – Forward one character.
M-f – Forward one word.
M-b – Backward one word.

C-a – Go to the beginning of the line.
C-e – Go to the end of the line.
M-< – Go to the beginning of the file.
M->- Go to the end of the file.
C-d – Delete a character.
M-d – Delete a word.
C-k – Kill (cut).
C-y – Yank (paste).
C-x u – Undo.
C-s – Start a forward search.
C-r – Start a reverse search.

Emacs Repeat Commands:

C-u N <command> – Repeat <command> N times.
For example:
C-u 10 C-d

Find Files and Directories

find [path] [expression]

Day 2 Notes on “Learn Linux in 5 Days and Level Up Your Career” by Jason Cannon on Udemy

Day 2

1. Linux Directory Structure:

Most commonly used top-level directories:

/ “Root”, the top of the file system hierarchy
/bin Binaries and other executable programs
/etc System configuration files
/home Home directory (various user files)
/opt Optional or third party software
/tmp Temporary space, typically cleared on reboot
/usr User related programs, libraries, and docs.
/var Variable data, most notably log files

Applications that are not shipped with the OS are typically installed in:


Comprehensive Directory Listing

/boot Files needed to boot the operating system.
/cdrom Mount point for CD-ROMs.
/cgroup Control Groups hierarchy.
/dev Device files, typically controlled by the operating system and the system administrators.
/etc System configuration files.
/export Shared file systems.
/home Home directories.
/lib System Libraries.
/lib64 System Libraries, 64 bit.
/lost+found Used by the file system to store recovered files after a file system check has been performed.
/media Some Linux OS might use to mount removable media like CD-ROMs.
/mnt Used to mount external file systems.
/opt Optional or third party software.
/proc Provides info about running processes.
/root The home directory for the root account.
/sbin System administration binaries.
/selinux Used to display information about SELinux.
/usr/bin Binaries and other executable programs.
/usr/lib Libraries.
/usr/local Locally installed software that is not part of the base operating system.
/usr/sbin System administration binaries.
/var Variable data, most notably log files.
/var/log Log files.

Some Linux OS might use:

/srv Contains data which is served by the system.
/srv/www Web server files.
/srv/ftp FTP files.
/sys Used to display and sometimes configure the devices known to the Linux kernel.

Application directory structures:

Often install in:

Application installed in /opt


Examples of specific companies:



Variation applications sometimes follow:


Applications without given directory structure, so they are installed in a shared manner:


2. What is the Shell?

– The default interface to Linux
– A program that accepts your commands and executes those commands
– Also called a command-line interpreter

Command Line Interface (CLI)
Graphical User Interface (GUI)

The prompt:

[philipp@linuxsvr ~]$ Username and the Linux system I’m connected to. The dollar sign tells me that I am a normal user.
[philipp@linuxsvr ~]# The pound sign typically tells me that I am a superuser or the root account (administrator in a windows system).

Example prompts:

philipp@linuxsvr $
[philipp@linuxsvr /tmp]$
[16:45:51 linuxsvr ~]$

Tilde Expansion:

~philipp = /home/philipp
~pat = /home/pat
~root = /root
~ftp = /srv/ftp

3. Basic Linux Commands

ls – Lists directory contents.
ls -l – Long list of directory contents.
cd – Changes the current directory.
cd without any arguments takes you to your home directory.
pwd – Displays the present working directory.
cat – Concatenates and displays files.
echo – Displays arguments to the screen.
echo $PATH
man – Displays the online manual.
man ls – To learn more about the command: ls
Type q to exit the manual.
exit – Exits the shell or your current session.
clear – Clears the screen.

which command:

which – Locate a command.
which cat
which tac – Display content of file in reverse order.

--help command:

--help – Add to a command to get help.
-hls -h i.e.

man document navigation:

Enter key – Moves down line by line.
Space bar – Moves down an entire paoge.
Lower case ‘g’ key – Move to the top of the screen.
Capital ‘G’ key – Move to the bottom of the screen.
Lower case ‘q’ key – To exit/quit.

If you are not sure which command to use, you can search through the manual page using this command: man -k SEARCH_TERM.
For example: man -k calendar.


. – This directory.
.. – The parent directory.
cd - – Change to the previous directory.
/ – Directory seperator (forward slash).
echo $OLDPWD – Shows directory we have been previously in.
./cat – Execute something in current directory, i.e. cat (instead of the $PATH directory).

mkdir [-p] directory – Create a directory.
rmdir [-p] directory – Remove a empty directory.
rm -rf directory – Recursively removes everything in and below that directory. (Be careful, using rm -rf removes are not undoable!)
The part of the command in square brackets is optional.
[-p] – Parents, for example:
mkdir -p 1 directory1/directory2/directory3

The ls command:

ls -l – Detailed list of directory content.
Example details displayed:
Permission: -rw-rw-r--
Number of links: 1
Owner name: philipp
Group name: users
Number of bytes in the file: 10400
Last modification time: Sep 27 08:52
File name: sales.data
Hidden files are not displayed using ls, you need to use ls -a. Hidden files begin with a dot i.e.: .mydata.
Combine the ls command with command line arguments, for example ls to list files with -l to display a long list, and -a for all files, including hidden.
ls -l -a can also be written as ls -la

Use ls -F to reveal file types.
or ls -lF for a long listing format.
For example, ending with:
/ – A directory.
@ – A link (symbolic link to file).
* – An executable.

ls -t – List files by time.
ls -r – Reverse order.
ls -latr – Long listing including all files reverse sorted by time.
ls -R – Lists files recursively, in the current directory and any sub-directory below that.
ls -d – Only list directory names, not their content.
ls --color – Colorize the output.
Combination of -F and --color: ls -F --color

If there is a file with a space (try to avoid this), you can use double or single quotes to execute a command: ls -l "my notes.txt"

The tree command:

Similar to ls -R, but creates visual output:
tree -d – List directories only.
tree -C – Colorize output.
Combine -d and -C: tree -Cd

Day 1 Notes on “Learn Linux in 5 Days and Level Up Your Career” by Jason Cannon on Udemy

Day 1

1. Brief History:

Linux is an operating system and kernel, it is the layer between application and hardware.
Created by Linus Torvalds in 1991: Wish to run Unix on a PC.

2. Distributions:

Linux Operation System + Apps = Distribution or Flavor

Learn about all available distributions on distrowatch.com
Most common distributions:
Red Hat Enterprise Linux (RHEL)
SuSE Linux Enterprise Server (SLES)

3. Why Linux:

Runs on many platforms
Small footprint
Stable, reliable, secure
Great for servers
FOSS (Free/Open Source Software)

4. Let’s get started!

Download Virtual Box

Download and install VirtualBox for windows, here.
Select “Windows hosts”

Download 7-zip

Download and install 7-zip if you don’t have it: click here.
I selected the 64-bit file.

Download CentOS

Now, download the CentOS 8 virtual disk image from this link.
Select “CLICK HERE –> CentOS 8 – Gnome Desktop VDI – Download <– CLICK HERE” and extract the file.

Creating CentOS Virtual Machine

Open up VirtualBox and click new:
Name: centosdesktop
Type: Linux
Version: Red Hat (64-bit)

Click next, and enter the amount of memory (RAM) you want to use, I have 32GB, so I entered 8192 MB.

Tick the box: Use an existing virtual hard disk file and add the downloaded CentOS 8 virtual disk image downloaded earlier.
Click create.

Booting CentOS

To boot CentOS, click “Start” in the main menu of VirtualBox.

Sign in password will be: “adminuser”, just as the user name itself.
(If you are using another virtual disk image, the username and password should come with it)

You can find the terminal under Activities > Terminal, or type terminal in the prompt.

To exit the terminal type: exit and hit enter.

Connecting to a Linux System over the Network

This would most likely happen over the SSH protocol. (Secure Shell)

You need an SSH client and terminal emulator, for windows the most popular is PuTTY. Go here.

Download: “64-bit: putty.exe”, open file and run program.

In the Host (or IP address) field, type in the network name of your Linux server. In this example linuxsvr. (More on how to connect to a Linux system via the network on day 5)

Commands learned:

exit to exit out of the terminal.
ssh linuxsvr to create an ssh host called linuxsvr. (It is also possible to use an IP address instead)
uptime to get the time running the Virtual Machine i.e.

Sci Fi Demo Project [Unity & WebGL]

Sci Fi Demo Project >>> PLAY NOW <<<


This is the Sci Fi Demo Project by Jonathan Weinberger on Udemy.
Use the “E” button to pick up the coin from the vending machine and buy a weapon from the Shark also with “E” once you picked up the coin.
There is a destructible crate on the opposite side, in front of the fruit vendor.

In case you want to test play the game, here you go!

This is the project for you to download: Sci Fi Project

Galaxy Shooter [Unity & WebGL]

My first from idea to deployment Unity game project! >>> PLAY NOW <<<


Hi guys, I have been playing around and following some tutorials with Unity which truly is great fun and a fantastic way of learning to code in a playful way. Of course the game I have been developing following the Udemy tutorial by Jonathan Weinberger is very simple, but it was critical to see how the whole process could look like, from the idea of the game, asset management, programming the game, post-processing and building the game with WebGL and finally hosting it on GitHub (which I added, because I wanted to share the game with friends). So now I know how to go through this steps and am ready to look into adding complexity along the way.

In case you want to test play the game, here you go! I know its not perfect, and there is room for improvement, but the excitement holds up for at least 30 seconds (yay!).

This is the project for you to download: Galaxy Shooter