# HPC-GAP

GAP includes experimental code to support multithreaded programming in GAP,
dubbed HPC-GAP (where HPC stands for "high performance computing"). GAP and
HPC-GAP codebases diverged during the project, and we are currently working
on unifying the codebases and incorporating the HPC-GAP code back into the
mainstream GAP versions.

This is work in progress, and HPC-GAP as it is included with GAP right now
still suffers from various limitations and problems, which we are actively
working on to resolve. However, including it with GAP (disabled by default)
considerably simplifies development of HPC-GAP. It also means that you can
very easily get a (rough!) sneak peak of HPC-GAP. It comes together with the
new manual book called "HPC-GAP Reference Manual" and located in the `doc/hpc`
directory.

Users interested in experimenting with shared memory parallel programming in
GAP can build HPC-GAP by following the instructions from
<https://github.com/gap-system/gap/wiki/Building-HPC-GAP>. While it is possible
to build HPC-GAP from a release version of GAP you downloaded from the GAP
website, due to the ongoing development of HPC-GAP, we recommend that you
instead build HPC-GAP from the latest development version available in the
GAP repository at GitHub, i.e., <https://github.com/gap-system/gap>.


## HPC-GAP code in the GAP kernel

Some hints on how and where to find HPC-GAP specific code in the GAP kernel:

* The `hpcgap/src/` directory contains two files, `c_oper1.c` and `c_type1.c`,
  which are counterparts to their namesakes in `src/`; all these `c_*.c` files
  are generated by `make docomp`, which invokes the GAP compiler `gac` to
  compile `lib/oper1.g` respectively `lib/type1.g` into C code. The `c_*.c`
  files should never be manually edited.

* Code inside `src/hpc/` is mostly HPC-GAP specific. E.g. the code for dealing
  with threads, locks, guards, regions, atomic lists and records, etc., all can
  be found in here.

  Note that `src/hpc/serialize.{c,h}` and `src/hpc/traverse.{c,h}` might one
  day be used in regular GAP as well, but then they would be moved from
  `src/hpc/` to `src/`.

* Most of the remaining HPC-GAP code is inside of `#ifdef HPCGAP / #endif`
  blocks and similar constructs, and thus can be easily found by searching
  for `HPCGAP`.

* An exception to the previous rule is that `HashLock` and `HashUnlock` may be
  called in regular GAP, too, where they do nothing and are optimized away. This
  is for convenience, as typically code using these needs to call `HashUnlock`
  in multiple places, i.e., at every exit of a function, making it cumbersome
  to wrap each call into `#ifdef HPCGAP / #endif`.

* The interface to the Boehm garbage collector in `src/boehm_gc.c` is in theory
  not limited to HPC-GAP, but in practice it is, at least for now.


## HPC-GAP code in the GAP library

Normally, the GAP library is contained in the `lib` directory inside the
GAPROOT directory. But when GAP is compiled and run in HPC-GAP code mode, then
whenever a library file is to be loaded, it first looks for that file inside
`hpcgap/lib/`, and only if that fails, then it falls back to searching in
`lib`. This allows us to take a file from `lib`, duplicate it into `hpcgap/lib`,
and then apply HPC-GAP specific changes.

However, we try to keep such duplicated files to a minimum, as it is cumbersome
to maintain them: if `lib/foo.gi` is modified, then one must always remember to
also apply the same changes to its clone twin `hpcpgap/lib/foo.gi`.

Instead, it is preferable to place HPC-GAP specific code inside a conditional
using `IsHPCGAP`, like this:

```
  if IsHPCGAP then
     # perform HPC-GAP specific actions
  else
    # perform altrnative actions for regular GAP
  fi;
```

Note that `IsHPCGAP` is a global constant, set to either `true` or `false` by
the kernel before loading the library. Since GAP optimizes conditionals using
global constants away, the above code actually has zero performance overhead
when compared to the version not using it. I.e., `if IsHPCGAP` in GAP code
behaves similarly to `#ifdef HPCGAP` in C code.

As a caveat to this, code which needs to use `atomic` statements usually is
difficult to write using `if IsHPCGAP` without duplicating a lot of code. For
this reason, we allow using `atomic` statements in regular GAP, too, where
they have no effect and also cause zero overhead. Exploiting this, we allow
ourselves to use `atomic` outside of `if IsHPCGAP then ... fi` in some limited
cases inside the `lib/` directory. However, it is recommended to keep this to
a minimum, as the code becomes harder to understand and maintain for people
who are not familiar with HPC-GAP. Instead, it is preferable to find other
solutions, such as high-level abstractions which can be used uniformly by
regular GAP and HPC-GAP code, which encapsulate and hide the HPC specifics. An
example for that is `MemoizePosIntFunction`. See also
<https://github.com/gap-system/gap/issues/1889>.

As of the time this is written, only the following library files in `lib`
make use of `atomic` statements:

  1. `lib/coll.gd`
  1. `lib/coll.gi`
  1. `lib/filter.g`
  1. `lib/grppc.gi`
  1. `lib/helpdef.gi`
  1. `lib/info.gi`
  1. `lib/methwhy.g`
  1. `lib/oper.g`
  1. `lib/package.gi`
  1. `lib/profile.g`
  1. `lib/type.g`

Some further places with HPC-GAP specific code include the following:

* `hpcgap/lib/hpc/` and `hpcgap/lib/distributed/` contain all truly HPC-GAP
  specific code.

* `lib/hpc/` contains placeholders for some of the code in `hpcgap/lib/hpc/`,
   make it writing code which works in both regular GAP and HPC-GAP more
   convenient.
