Operating system run a lot of its operation in kernel. But what part of computer that load the kernel?

That part is called bootloader. Bootloader is usually located in BIOS (basic input output system). This bootloader then initialize the second boot loader that live in a fixed address of memory, this second bootloader then load the kernel. This two layer process is unified to one now, using UEFI (Unified extensible firmware interface) which is faster.

Bootloader doing basically three important tasks:
First, load the file containing the kernel to the memory.
Second, run diagnostic to determine if the machine is ready to boot up.
Third, initialize device controllers to main memory to help booting.

One of bootloader program example is GRUB. GRUB is used in linux. GRUB is flexible, it can be controlled by the configuration file to variation of way, even we can change the kernel we want to run. GRUB load compressed kernel to main memory and then decompressed it. This technique is used to speed up the booting process. And then it will create a temporary RAM file system to cover devices that needed, after booting finished it will swith to normal file system in the disk, this is done like this to speed up the booting process (again). After that it will start the other services needed, and show up the login page.

Every bootloader have different modes, like recovery mode or safe mode. This modes is used to diagnose hardware issue, fixing corrupt file system, or even reinstall the operating system.

Reference Silberschatz. A, Galvin. P. B, Gagne. G, Operating System Concepts Tenth Edition: 2018