Skip to main content

Kdump configuration on Oracle Linux




Kdump configuration on Oracle Linux


We’ll discuss the configuration of kdump on an Oracle Linux server.All steps and procedures explained and performed are based on Oracle Linux release 7.



What is kdump?
Kdump is the crash dump mechanism of the Linux kernel which allows us to save the system memory contents for analysis. Oracle recommends that you enable the kdump feature as it helps Oracle Linux support to find the root cause whenever a server panics or crashes.


To help identify the cause of the crash in case of a system failure, a memory image called the vmcore is created by kdump. An exclusive system memory portion will be requisite for kdump to be enabled. Being exclusive, this memory is not available for other purposes. Kdump utilizes kexec to boot to a second kernel whenever the system crashes.


What is kexec?
Kexec is a quick boot mechanism that allows the Linux kernel to boot from within a kernel context that is already running without going through the bootloader stage.


Pre-requisites for kdump are:

  • That the kexec-tools package be installed on the server.


  • And a certain amount of memory be reserved for kdump. This is for kexec to boot in to the secondary kernel.


  • And a path be specified for the memory dump to be stored and it should have sufficient free space.


It is also necessary that the kdump service be running.


kdump mainly uses 3 files; the grub configuration file. This is where we specify the crash kernel parameter to reserve memory for kdump. Next the /etc/kdump.conf file, which is the kdump configuration file. This is where we set the location for storing the dump file and also the core collector filtering level that will be the make dumpfile command.



And the default behavior to take if the dump fails. And it also relies on the kdump initramfs file, which will be present under the /boot directory.



Let's go through a quick demo on how to configure kdump and generate a dump from it.



Firstly, we need to make sure that the kexec-tools package is installed on the server. As it is not installed, so let's go ahead and do the installation first. Now the package is installed.



Next we have to verify the crashkernel parameter in the grub configuration file. This is done via the /etc/default/grub file, since we are using grub2. Here we set the crashkernel parameter and reserve a certain amount of memory for kdump.


The value for crashkernel can be either auto or a specific amount of memory. We can see that it is set to 256M at an offset of 64M on the server. For larger servers with higher amount of physical memory, for example 128GB of physical memory, it is recommended to set crashkernel to 512M. And similarly, for even larger servers, a higher amount of memory needs to be reserved for kdump.


Once the crashkernel parameter is added to the /etc/default/grub file, the grub configuration file needs to be rebuilt. For that, we use the grub2-mkconfig command. The grub entries for each installed kernel was updated.



Let's see the grub configuration file. We see that the crashkernel parameter was added to the kernel entries in the grub configuration file. Once the grub configuration is updated, the server needs to be rebooted for the changes to take effect. I have already rebooted the server previously. Next, we have to configure kdump service, which is done via the /etc/kdump.conf file.



Here we see that the path parameter is set to /var/crash. This is the default path. This is used to specify the location for the memory dump to be stored. It is essential that the location have sufficient space.



Next the core collector needs to be configured and we’ll be using makedumpfile as our core collector. The -d option is for the dump_level parameter. Dump level specifies the unnecessary pages for analysis. These pages are not copied to the memory dump.  The message_level parameter is for setting the message types that need to be collected.


Users can restrict outputs printed by setting the message level. Default value for message level is 7. Here we are setting it to 1.




After making changes to the kdump configuration file, the kdump service needs to be restarted. While restarting the kdump service, the initramfs file is rebuilt to update the changes. It will be present under the /boot directory. Let's see the file.



The kdump initramfs file is generated. To ensure that the kdump configuration is proper and a vmcore will be generated when the server crashes, we can trigger a manual crash of the server and verify that the dump is being generated.



This is done by echoing the 'c' character to the / proc / sysrq-trigger file. Sysrq is a magic key, and these keys are a combination of keys that can be pressed, and no matter what else it does, the kernel can respond to it unless it is absolutely locked.



Echoing a character 'c' to the sysrq-trigger file causes the server to crash by a null pointer dereference. Before triggering a crash, make sure that a downtime is available for the activity in case the server is a live production system.



Let's crash the server. Now we are in the serial console of the server. We can see that the dump is being collected and that is has completed collecting it. Once the dump is collected the server goes for a normal reboot. The time required to collect the dump varies based on the amount of memory that needs to be dumped.



Now the system is rebooting and it is up now. Let's check the path where we had configured the dump to be collected. Check under /var/crash. We see that a directory with timestamp was created. There are two files under it, that is the vmcore file and the vmcore-dmesg file.



We find that it is a data file. And the size is considerably small as the server has only 4GB of memory. So the kdump configuration was successful and a memory dump is generated. In case the server crashed at any time, we can be sure that the vmcore will be generated.


Ending Comment:

You can upload the vmcore file to Oracle support for analysis. Some useful links and few ‘my oracle support knowledge documents’ can be accessed by logging in to support.oracle.com.  Please go through them for more information on kdump and known issues. 


That ends our short writing on configuring kdump on an Oracle Linux server.


Labels:

Comments

Post a Comment

Please do not add any spam link in the comment box.

Popular posts from this blog

Which one is better against coronavirus: Hand sanitizer or Soap?

With the world crippled by coronavirus and economies stagnated, the biggest question that comes to mind is the protection against this deadly pandemic. So, we are going to talk about  Which one is better against coronavirus : Hand sanitizer or Soap? If you look closely, your hands are anything but soft. With bumps and gaps, folds and meanders, there are many places where a virus can hide. The virus can infect you if you then touch your face. However, there are two incredibly simple ways to avoid this: hand sanitizer, and soap and water.  So what's the best way to go about? The coronavirus that causes COVID-19, like other viruses, has a outer protective surface made of a lipid bilayer. The head of these lipids are attracted to water, while its tail is repelled by it. They have a pin-shaped structure. Lipids form a shell in water-rich environments with the head outside and the tail inside. Due to the joint reaction with water, the lipids adhere to one...
Post a Comment
Read more

Preparing Krispy Kreme donut at home

So how possible is it to recreate  Krispy Kreme donuts  at home? You know, they got the signature taste, perfect circle shape, the glaze on top. I've described to you what they are, but can we make them at home? That is the question. So making  donuts  is like a combination of several different skills and techniques, and it's making dough and shaping it and deep-frying . And I've made jelly donuts , and they were some of the best. So expectations are through the roof right now about these as well. I need the ingredients for the dough, and we'll go from there. Ingredients: 300 millilitres of whole milk Seven grams of instant yeast Two large eggs 113 grams of melted butter, that's cooled too  50 grams of granulated sugar teaspoon of salt 535 grams of bread flour     Method of Preparing Krispy Kreme donut I'm just going to heat up that milk, gotta be warm to touch. So the yeast goe...
Post a Comment
Read more

Nike (NKE) Stock Analysis

W hat ’ s up investors , another stock analysis this time we're taking a look at Nike ticker symbol NK E . I do own shares of Nike , but it should not impact how I analyze this company . L ike I said , they just reported earnings today . S o it's actually really exciting . S o I'll go through those we've got the three months ended. I t was our Q4 technically . S o they've got their 12-month ended , and Nike's a really interesting company because we've got this whole C OVID- 19 thing happening . Nike had all their stores closed , they had sports basically around the world not being played . S o Nike , out of all companies , should be very heavily impacted by the virus . Y ou see revenues down 38% . S o after Nike posted these earnings , only down about 3% , which really takes us back to a price that we saw like last week or so . S o not heavy action in Nike , we'll see if the earnings that we go ...
Post a Comment
Read more