View source for Yocto build on a cluster with IceCC

== Yocto build on a cluster with IceCC ==

This section is copied from https://www.openembedded.org/wiki/Using_IceCC

== IceCC and OE ==
It is possible to compile on a cluster of machines with OE, and quite easily so. To do so add the following to your local.conf:

 ICECC_PARALLEL_MAKE = "-j 24"
 INHERIT += "icecc"

General consensus is that ICECC_PARALLEL_MAKE should be between 2x to 4x the number of cores your CPU has. Too many and your PC will be unusable, too few and you won't keep the cluster busy.

=== Icecc and sstate interaction ===

Inheriting icecc will change the hashes for all tasks, so you cannot share sstate between builds that have icecc inherited and those that don't. The recommended solution is add support for icecream to all builds as follows:

 INHERIT += "icecc"
 ICECC_DISABLED ??= "1"

Now, users that want to use icecream can enable it by setting ICECC_DISABLED = "0", but everyone can still share sstate.

=== Recipe build failures ===

Many recipes may fail to build under icecream for a variety of reasons. If you find a recipe that does so, you can add it to ICECC_USER_PACKAGE_BL in local.conf.

We try to keep the system blacklist (ICECC_SYSTEM_PACKAGE_BL) up to date, but packages sometimes get missed (or aren't part of oe-core and don't get tested at all).

Note that any recipe that disables PARALLEL_MAKE by setting it to "" will automatically have icecream disabled.

=== Dedicating a scheduler ===
By default, icecream will pseudo-randomly assign one of the compile nodes to be the scheduler (the node that decides where all the compiles get distributed). In some cases, it may be desirable
to dedicate a specific computer to be the scheduler. You may want to do this if:
* The cluster is large. The CPU time required to do scheduling can be noticable
* The cluster has a lot of mobile devices (e.g. laptops). If one of these happens to be assigned to be the scheduler and then goes offline, it can take an while to recover.

=== Network Bandwidth Considerations ===
Icecream performance is very dependent on the speed of the network between the cluster nodes. As a rule of thumb, 1 Gpbs or better is ideal, and with anything less than 100 Mbps, it's probably not worth using at all. Wifi is dubiously useful at any speed.

=== Credits ===
Icecream support was originally implement by zecke, improved by Ifaistos, and most recently by Joshua Watt (JPEW)

=== Utilities ===
There are several useful utilities when dealing with icecream:
* [https://github.com/icecc/icemon icemon] - A gui tool for monitoring the cluster
* [https://github.com/JPEWdev/icecream-sundae icecream-sundae] - A command line tool for monitoring the cluster

[[Category:User]]
[[Category:FAQ]]

=== More Information ===
* [https://www.youtube.com/watch?v=VpK27pI64jQ&list=PLbzoR-pLrL6ol7Cf1g_4rsCda23OiLh8d&index=26 Sweeten Your Yocto Build Times with Icecream] - Joshua Watt, Embedded Linux Conference North America 2019 ([https://drive.google.com/file/d/1SKI9P86fx-IHLoststTn9kh22xaln6AA/view?usp=sharing slides])