1- What is virtual Memory? Swap space. It's a area on the storage (hard disk) wich uses if RAM is low.

Some times it's calling paging space. 1.1- what is memory pressure? The Linux VM code tries to use up spare memory for cache, thus there is normally little free memory on a running system. The intent is to use memory as efficiently as possible, and that cache should be easily recove rable when needed. When the system needs a page and there are insufficient available the system wil l trigger reclaim, that is it will start the process of identifying and releasing currently in-use pages (kswapd and direct reclaim). 1.2- Is there a command that would show the system to be in such a state? The vmstat or free commands can be used to monitor system virtual memory activit y - /proc/meminfo, /proc/stat, /proc/*/stat, /proc/sys/vm/* It shows the amount of space allocated on the swap device, the number of unused pages in RAM, the sizes of the buffer and page cache and the number of blocks swapped in and out over a given period of time. 1.3- what is a page fault? A page fault occurs when a process must access the disk or swap device to satisf y the page request. 1.4- what is the oom-killer? when is it triggered? The OOM killer subsystem is an algorithm which is invoked when the system runs o ut of paging space. When the system runs out of memory, you will typically see messages either from the OOM killer, or page allocation failures. These are typically symptoms that either your workload is to high for the machin e or something is wrong. If the workload does not fit into RAM + SWAP, then you are going to run out of m emory. If it does not fit into RAM, then it will probably perform badly, but should still work. 1.5- Can the sysadmin force the OOM killer to start? Yes we can, by sending SysReq commands "m" to dump current memory info to your c onsole and "f" to call oom_kill to kill a memory hog process. # echo "f" > /proc/sysrq-trigger 1.6- Can the kernel swap to a ramdisk? Yes. # mkswap /dev/ram0 # swapoff -a # swapon /dev/ram0 1.7- How can a 32-bit system use more than 4GB of RAM? It possible if you will install PAE kernel (Physical Address Extention) 2- How does one list the pci peripherals connected to the system? # lspci 2.1- For one particular PCI peripheral, how can one know if it is supported by a ny of the modules provided in linux? # lsmod # cat /lib/modules/2.6.*.el5/modules.pcimap 2.2- what other methods/commands can be used to find out which devices are conne cted to the computer (including disks and other peripherals in general) # dmesg

# # # # # # # # #

dmidecode lsusb lshal ethtool cat /proc/cpuinfo cat /proc/memory fdisk, parted ls -l /dev/ hdparm

4- What is a filesystem? It's system for hard drives to storage of information. With hierarchy of catalog s, files etc. Different file systems have different features and restrictions. 4.1- What are the main structures used in a filesystem? FHS, inodes or fact that in Linux, everything is configured as a file. Not clear a bit. 4.2- Do all filesystems implement all the file and directory access system calls ? When making a directory for instance, do filesystems share the same code in th e syscall? Yes. Filesystem will share the same code (inode) in the syscall if the directory instanced hardly (hard link) linked, no in case of symbolic link instances. 4.3- Is it possible to have a hard link spanning two files in different filesyst ems? What is the reason? No. Hard link possible to create only in one filesystem. In other reason you sho ud use soft link. Hard link works with inode. Since the inodes are unique to eac h filesystem, you won't be able to reference an inode on a different disk or eve n a different filesystem. 4.4- can you use 8k block ext3 filesystems on x86 machines? No. x86 filesystem block is always 4k by default and never larger than the size of a memory page. It's possible to extend in 8k on Intel Itanium for example. 4.5- What are the reasons why some applications insist on accessing raw devices? Because they need bypass the system buffer cache 4.6- What type of file system would be most suitable for a character device? Raw filesystem. Because character devices are read and written without buffering . 4.7- What is an access control list? How is it used in Linux? An access control list (ACL) is a table that defines the user privileges policy of the operating system. In Linux ACL on files and directories like drwx-rw-r-x, where: r or 4 - read permission w or 2 - write permission x or 1 - execute permission d - directory b - block device - file + flags sgid, suid, sticky bit 5- What is the main binary executable format used in linux? Can you describe the most important components of this format? ELF (Executable and Linkable Format). It is used for both regular binary executa bles, shared libraries, object code (.o files) and core dumps, because their internal structure is quite similar. An ELF file s tarts with an ELF header, followed by an image header and the actual data. This

is divided into sections, a concept familiar to any Assembly language programmer . 5.1- what is linking? Linking is the process of combining various pieces of code and data together to form a single executable that can be loaded in memory. 5.2- what is a shared library? The shared library is a library/program that allow executables to dynamically ac cess external functionality at run time and thereby reduce their overall memory footprint by bringing functionality in when it's needed. 5.3- How can one see all the shared libraries which a binary is linked to? ldd -v "way to file" e.g. ldd -v /bin/bash 5.4- A sysadmin removed by accident all the links in the /lib directory leaving the binaries, now many applications do not load. What is the best way of finding the links through which a library should be accessed? Copy all files from other systems by hand or make links from /usr/lib 5.5- How can one recover from the situation on (5.4)? # ldconfig /lib 5.6- Is it possible to override the functions defined in a shared library when r unning an application? The current Linux shared libraries are much more flexible and sophisticated that permit us to override the specific functions in a library when executing a part icular program. It can be done without messing up with the library source code o r having root permissions in order to install a patched version of the library. And it makes sense to do that, for debugging purposes or transparent extensions. 5.7- What kind of memory protection is needed in order for the operating system to correctly implement shared libraries? Shared libraries are designed with a technique for placing library functions int o a single unit that can be shared by multiple processes at run time. This techn ique save both disk space and RAM. But I am not sure about as I had no chance to deal to much with shared libraries in my experience. 6- A sysadmin is operating a linux system with 6 network cards, being that only eth0 is plugged in and working. Sysadmin does a few commands: # ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:11:44:43:63:E1 inet addr:192.168.1.104 Bcast:192.168.1.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2231 errors:0 dropped:0 overruns:0 frame:0 TX packets:2522 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:100 RX bytes:780859 (762.5 KiB) TX bytes:373261 (364.5 KiB) Base address:0x8000 Memory:c0220000-c0240000 # route -n Kernel IP routing table Destination Gateway 192.168.1.0 0.0.0.0 0.0.0.0 192.168.1.1 # iptables -L Chain INPUT (policy ACCEPT)

Genmask 255.255.255.0 0.0.0.0

Flags Metric Ref U 0 0 UG 0 0

Use Iface 0 eth0 0 eth0

target

prot opt source

destination destination destination

Chain FORWARD (policy ACCEPT) target prot opt source Chain OUTPUT (policy ACCEPT) target prot opt source Questions:

6.1- What is the IP address of this system and the router it is configured to us e? IP address of this system is 192.168.1.104. Default gateway (router) is 192.168. 1.1 6.2- How can one know the speed of the interface eth0? # mii-tools # ethtool eth0 | grep -i speed 6.3- How can one know the pci card that is currently eth0 and the driver it's cu rrently using? # lspci | grep -i net # cat /etc/modprobe.conf # lsmod | grep e1000 or # ethtool -i eth0 driver: e1000 version: 7.3.21-k4.1-NAPI firmware-version: N/A bus-info: 0000:02:00.0 6.4- Sysadmin left a tcpdump running as follows: # tcpdump -n -nn -i eth0 on one terminal, while issuing commands on another terminal. With the following output from tcpdump, can you tell what commands were run, and if they succeeded or failed? 13:43:35.087797 IP 192.168.1.104 192.168.1.1: ICMP echo request, id 10254, seq 1 , length 64 13:43:35.089529 IP 192.168.1.1 192.168.1.104: ICMP echo reply, id 10254, seq 1, length 64 13:43:36.087483 IP 192.168.1.104 192.168.1.1: ICMP echo request, id 10254, seq 2 , length 64 13:43:36.088225 IP 192.168.1.1 192.168.1.104: ICMP echo reply, id 10254, seq 2, length 64 14:04:09.015376 IP 192.168.1.104.51276 192.168.1.1.33434: UDP, length 40 14:04:09.015759 IP 192.168.1.104.33382 192.168.1.1.33435: UDP, length 40 14:04:09.015820 IP 192.168.1.104.45147 192.168.1.1.33436: UDP, length 40 14:04:09.015877 IP 192.168.1.104.33561 192.168.1.1.33437: UDP, length 40 14:04:09.015974 IP 192.168.1.104.42414 192.168.1.1.33438: UDP, length 40 14:04:09.016034 IP 192.168.1.104.56430 192.168.1.1.33439: UDP, length 40 14:04:09.016091 IP 192.168.1.104.56784 192.168.1.1.33440: UDP, length 40 14:04:09.016147 IP 192.168.1.104.59999 192.168.1.1.33441: UDP, length 40 14:04:09.016207 IP 192.168.1.1 192.168.1.104: ICMP 192.168.1.1 udp port 33434 un reachable, length 76 14:04:09.016281 IP 192.168.1.104.35841 192.168.1.1.33442: UDP, length 40 14:04:09.016953 IP 192.168.1.1 192.168.1.104: ICMP 192.168.1.1 udp port 33435 un reachable, length 76

14:04:09.017452 IP 192.168.1.1 192.168.1.104: ICMP 192.168.1.1 udp port 33436 un reachable, length 76 14:04:09.017952 IP 192.168.1.1 192.168.1.104: ICMP 192.168.1.1 udp port 33437 un reachable, length 76 14:04:09.018452 IP 192.168.1.1 192.168.1.104: ICMP 192.168.1.1 udp port 33438 un reachable, length 76 14:04:09.019201 IP 192.168.1.1 192.168.1.104: ICMP 192.168.1.1 udp port 33439 un reachable, length 76 14:51:43.018084 IP 192.168.1.104.42034 192.168.1.1.23: S 2650503081:2650503081(0 ) win 5840 <mss 1460,sackOK,timestamp 5378506 0,nop,wscale 7> 14:51:43.018890 IP 192.168.1.1.23 192.168.1.104.42034: R 0:0(0) ack 2650503082 w in 0 14:51:48.017352 arp who-has 192.168.1.1 tell 192.168.1.104 14:51:48.017600 arp reply 192.168.1.1 is-at 00:1e:35:3e:1c:41 14:51:55.249807 IP 192.168.1.104.33796 192.168.1.1.80: S 2660968684:2660968684(0 ) win 5840 <mss 1460,sackOK,timestamp 5390740 0,nop,wscale 7> 14:51:55.251545 IP 192.168.1.1.80 192.168.1.104.33796: S 2605137501:2605137501(0 ) ack 2660968685 win 5792 <mss 1460,sackOK,timestamp 574003227 5390740,nop,wscal e 7> 14:51:55.251580 IP 192.168.1.104.33796 192.168.1.1.80: . ack 1 win 46 <nop,nop,t imestamp 5390742 574003227> 14:51:55.251983 IP 192.168.1.104.33796 192.168.1.1.80: P 1:174(173) ack 1 win 46 <nop,nop,timestamp 5390742 574003227> 14:51:55.252543 IP 192.168.1.1.80 192.168.1.104.33796: . ack 174 win 46 <nop,nop ,timestamp 574003228 5390742> 14:51:55.289779 IP 192.168.1.1.80 192.168.1.104.33796: P 1:344(343) ack 174 win 46 <nop,nop,timestamp 574003265 5390742> 14:51:55.289839 IP 192.168.1.104.33796 192.168.1.1.80: . ack 344 win 54 <nop,nop ,timestamp 5390780 574003265> 14:51:55.291768 IP 192.168.1.1.80 192.168.1.104.33796: F 344:344(0) ack 174 win 46 <nop,nop,timestamp 574003267 5390780> 14:51:55.309637 IP 192.168.1.104.33796 192.168.1.1.80: F 174:174(0) ack 345 win 54 <nop,nop,timestamp 5390800 574003267> 14:51:55.310262 IP 192.168.1.1.80 192.168.1.104.33796: . ack 175 win 46 <nop,nop ,timestamp 574003286 5390800> # ping 192.168.1.1 13:43:35.087797 IP , length 64 13:43:35.089529 IP length 64 13:43:36.087483 IP , length 64 13:43:36.088225 IP length 64 SUCCEEDED -c2 192.168.1.104 192.168.1.1: ICMP echo request, id 10254, seq 1 192.168.1.1 192.168.1.104: ICMP echo reply, id 10254, seq 1, 192.168.1.104 192.168.1.1: ICMP echo request, id 10254, seq 2 192.168.1.1 192.168.1.104: ICMP echo reply, id 10254, seq 2,

# traceroute 192.168.1.1 14:04:09.015376 IP 192.168.1.104.51276 192.168.1.1.33434: UDP, 14:04:09.015759 IP 192.168.1.104.33382 192.168.1.1.33435: UDP, 14:04:09.015820 IP 192.168.1.104.45147 192.168.1.1.33436: UDP, 14:04:09.015877 IP 192.168.1.104.33561 192.168.1.1.33437: UDP, 14:04:09.015974 IP 192.168.1.104.42414 192.168.1.1.33438: UDP, 14:04:09.016034 IP 192.168.1.104.56430 192.168.1.1.33439: UDP, 14:04:09.016091 IP 192.168.1.104.56784 192.168.1.1.33440: UDP, 14:04:09.016147 IP 192.168.1.104.59999 192.168.1.1.33441: UDP, 14:04:09.016207 IP 192.168.1.1 192.168.1.104: ICMP 192.168.1.1 reachable, length 76 14:04:09.016281 IP 192.168.1.104.35841 192.168.1.1.33442: UDP, 14:04:09.016953 IP 192.168.1.1 192.168.1.104: ICMP 192.168.1.1

length 40 length 40 length 40 length 40 length 40 length 40 length 40 length 40 udp port 33434 un length 40 udp port 33435 un

reachable, length 76 14:04:09.017452 IP 192.168.1.1 reachable, length 76 14:04:09.017952 IP 192.168.1.1 reachable, length 76 14:04:09.018452 IP 192.168.1.1 reachable, length 76 14:04:09.019201 IP 192.168.1.1 reachable, length 76 SUCCEEDED

192.168.1.104: ICMP 192.168.1.1 udp port 33436 un 192.168.1.104: ICMP 192.168.1.1 udp port 33437 un 192.168.1.104: ICMP 192.168.1.1 udp port 33438 un 192.168.1.104: ICMP 192.168.1.1 udp port 33439 un

# telnet 192.168.1.1 23 14:51:43.018084 IP 192.168.1.104.42034 192.168.1.1.23: S 2650503081:2650503081(0 ) win 5840 <mss 1460,sackOK,timestamp 5378506 0,nop,wscale 7> 14:51:43.018890 IP 192.168.1.1.23 192.168.1.104.42034: R 0:0(0) ack 2650503082 w in 0 FAILED # wget 192.168.1.1 or telnet 192.168.1.1 80 and HEAD / HTTP/1.0 14:51:48.017352 arp who-has 192.168.1.1 tell 192.168.1.104 14:51:48.017600 arp reply 192.168.1.1 is-at 00:1e:35:3e:1c:41 14:51:55.249807 IP 192.168.1.104.33796 192.168.1.1.80: S 2660968684:2660968684(0 ) win 5840 <mss 1460,sackOK,timestamp 5390740 0,nop,wscale 7> 14:51:55.251545 IP 192.168.1.1.80 192.168.1.104.33796: S 2605137501:2605137501(0 ) ack 2660968685 win 5792 <mss 1460,sackOK,timestamp 574003227 5390740,nop,wscal e 7> 14:51:55.251580 IP 192.168.1.104.33796 192.168.1.1.80: . ack 1 win 46 <nop,nop,t imestamp 5390742 574003227> 14:51:55.251983 IP 192.168.1.104.33796 192.168.1.1.80: P 1:174(173) ack 1 win 46 <nop,nop,timestamp 5390742 574003227> 14:51:55.252543 IP 192.168.1.1.80 192.168.1.104.33796: . ack 174 win 46 <nop,nop ,timestamp 574003228 5390742> 14:51:55.289779 IP 192.168.1.1.80 192.168.1.104.33796: P 1:344(343) ack 174 win 46 <nop,nop,timestamp 574003265 5390742> 14:51:55.289839 IP 192.168.1.104.33796 192.168.1.1.80: . ack 344 win 54 <nop,nop ,timestamp 5390780 574003265> 14:51:55.291768 IP 192.168.1.1.80 192.168.1.104.33796: F 344:344(0) ack 174 win 46 <nop,nop,timestamp 574003267 5390780> 14:51:55.309637 IP 192.168.1.104.33796 192.168.1.1.80: F 174:174(0) ack 345 win 54 <nop,nop,timestamp 5390800 574003267> 14:51:55.310262 IP 192.168.1.1.80 192.168.1.104.33796: . ack 175 win 46 <nop,nop ,timestamp 574003286 5390800> SUCCEEDED 6.5- Still with the tcpdump running, sysadmin runs the command: # ping 192.168.1.104 -c3 PING 192.168.1.104 (192.168.1.104) 56(84) bytes of data. 64 bytes from 192.168.1.104: icmp_seq=1 ttl=64 time=0.082 ms 64 bytes from 192.168.1.104: icmp_seq=2 ttl=64 time=0.064 ms 64 bytes from 192.168.1.104: icmp_seq=3 ttl=64 time=0.057 ms --- 192.168.1.104 ping statistics --3 packets transmitted, 3 received, 0% packet loss, time 1999ms rtt min/avg/max/mdev = 0.057/0.067/0.082/0.014 ms However the tcpdump does not capture any of the traffic. The sysadmin runs the c ommand again in flood mode, and still nothing is captured. How can one tell if t his is a problem or not, and what files or commands could be able to prove the p revious statement?

We should listen for network traffic on loopback interface not on eth0 (our case ). # tcpdump -nni lo 7- The directory /data is mounted via NFS: 192.168.1.105:/data on /data type nfs (rw,soft,intr,addr=192.168.1.105) 7.1- Even though permissions are set correctly in the local system, root can not create files on it. What are the probable causes of this happening? Perhaps in /etc/exportfs seted parameter root_squash and we shoud switch off to no_root_squash. 7.2- What are the main differences between NFSv2, NFSv3 and NFSv4? NFSv2 - originally operated entirely over UDP, does not support safe asynchronou s writes (it writes syncronous) and has a poor error handling than NFSv3; it als o does not support 64-bit file sizes and offsets, not allowing clients to access more than 2Gb of file data. NFSv3 - uses both UDP and TCP protocols, depends on portmapper, rpc.mountd, rpc. lockd, rpc.statd, it is stateless, so an additional Network Lock Manager (NLM) p rotocol, an auxiliary protocol for file locking is required to support locking o f NFS-mounted files READ/WRITE. Also NLM is stateful in that the server LOCKED k eeps track of locks, it asynchronous writes on the server. NFSv4 - uses only TCP protocol, designed for internet use, it is stateful. Locki ng operations(open/read/write/lock/locku/close) are part of the protocol proper. NLM is not used by NFSv4. 7.3- What daemons are necessary to be running on an NFS server if a client wants to mount an NFSv3 share? rpcbind (rhel6), lockd, and rpc.statd chkconfig chkconfig chkconfig chkconfig portmap on nfs on nfslock on netfs on

7.4- To mount nfs4 shares it is not necessary to have all the daemons necessary for NFSv3, why is that? See the answer from 7.2 section. 7.5- How can one list what port is being used by each rpc daemon on a system? # rpcinfo -p 7.6- By looking at the port numbers for nfsd, one can see that nfs2, nfs3 and nf s4 all share the same port. How does one differentiate what version of the proto col will be mounted? Can you provide a packet dump showing exactly this request? 7.7- How can one show the statistics of NFS calls made to a server? # nfsstats -s 7.8- the /data mount actually comes from a cluster. The node that was serving /d ata had a sudden power failure. The virtual IP 192.168.1.105 migrated to another node, but the mount /data is now showing wrong data. When doing an `ls /data' o ne lists another directory exported by the second cluster node. What could be ha ppening in this case?

7.9- An NFS server is very loaded, running 8 nfsd threads. How can one prove to the sysadmin that it is necessary to increase the number of NFS threads? 8- from the output of this vmstat 1, what can you say about this system during v mstat 1 was running? procs r b 6 15 0 0 5 13 0 5 10 0 2 13 0 5 16 0 0 3 16 0 8 17 0 -----------memory---------- ---swap-- -----io---- --system-- -----cpu----swpd free buff cache si so bi bo in cs us sy id wa st 1635256 11096 124 8088 584 176100 1064 176104 1257 1923 21 79 0 1711616 1764100 1835916 1939504 1981832 8548 8512 8528 9104 8588 128 132 120 112 120 144 8156 8260 7836 48 76360 0 52484 92 69424 280 76368 439 375 26 70 4 0 256 52484 720 882 25 74 0 0 384 69424 544 993 24 76 0 0 396 102796 491 995 20 80 0 224 40608 447 547 23 65 12 0 580 76960 588 755 25 75 0 0

8168 128 102796 7700 128 40608 7960 92 76960

2058792 14376

9- A user runs a bash script and gets a segfault $ ./script.sh Segmentation fault The user notices that changing some of the values inside a loop in the bash scri pt prevents the segfault from happening. 9.1- Can the logic of a script lead it to a segfault? No. 9.2- What is the best way of instrumenting a script so that it doesn't segfault? 9.3- What is the best way to find out what happened in the segfault? Use gdb to track exact source of problem. 9.4- Can you create a very small bash script which segfaults? 10- User is running a process, but it seems to be stuck, when attaching strace t o the pid, strace shows an unfinished system call. 10.1- What is the meaning of an unfinished syscall? When the system call is being executed and meanwhile another one is being called from a different thread/process then strace try to preserve the order of those events and mark the ongoing call as being unfinished. 10.2- What is the best approach to understand what is happening in this process? strace -f 10.3- How can one tell exactly where the process is stuck, or how to debug the p roblem further? A process is said to be stuck when it cannot proceed because it is waiting for a n event that cannot, or does not, occur. So, if we want to find where the process is stuck we should put that in debug mo de by creating a break points.

We might want to consider running it from a debugger, instead of trying to attac h to it at runtime. 11- User is running a process, but it seems to be stuck, when attaching strace t o the pid, strace shows nothing at all. 11.1- how can one understand what is happening and the status of the process? We can test it by using links command: # links http://google.com # ps aux | grep links kripton 15918 0.0 0.0 tp://google.com 4976 1928 pts/1 S+ 12:22 0:00 links ht

# strace -f -e poll,select,connect -p 15918 11.2- What can one do to debug this case even further? 12- Superuser runs 'sync' on a linux system, but this command never returns, doi ng 'ps auxw | grep sync' the sysadmin notices that it is in 'D' state. Can the s ysadmin kill this process? The sysadmin tried to strace the process, which only showed the unfinished sync() syscall. Supposing that a support call will be opened to find the root cause of this issu e, what kind of information would you try to obtain, and why? The process with flag D is uninterruptable sleep and basically can not be killed by users and/or admins. Status uninterruptable means that process is performing so-called critical task, the signals do not stop the process or alter the behavior and it means also that the process is holding a semaphore or a crit ical system resources. The only way to kill the process in state D is reboot of the machine. 13- By accident a sysadmin removed one of the disks belonging to an active VG. W hen rebooting the system they notice that the system does not boot anymore. The disk that was removed was part of the VG but contained no data. What is the proc edure to bring the machine back to operation and the VG back with its LVs? Boot into rescue mode # lvm vgscan # lvm vgchange --removemissing vgname # lvm vgextend --restoremissing vgname pvname (if the removed disk was readed) 14- Write a shell command that lists all the RPMs installed in a system sorted b y package size, largest package first (use dpkg and related commands if you're n ot familiar with RPM and related commands) # rpm -qa --queryformat '%10{SIZE}\t%{NAME}-%{VERSION}-%{RELEASE}.%{ARCH}\n' | s ort -rn 15- A sysadmin just added a new TAPE drive to his system. The system already ha d a drive called /dev/st0, which works fine. The sysadmin added the tape drive a nd created a /dev/st1 for it: # ls -l /dev/st1 brw-r--r-- 1 root root 8, 1 Mar 2 23:16 /dev/st1 The sysadmin loads a tape into the drive and runs a dd, to write his 3G file ont o it, which should fit as the tape supports 200Gb.

# dd if=/tmp/backup.tar of=/dev/st1 bs=32k However the dd finishes with an error saying that the device is full at 250Mb. 15.1- Is it a problem caused by the block size of the dd? Are there any situatio ns where the block size used can change the behaviour of a drive? 15.2- Can you explain why this is not working as expected? 15.3- The sysadmin rebooted the system, and now the boot loader is not working p roperly, and GRUB complains about a problem at stage 1.5. What should one do? The GRUB problem at stage 1.5 is one of the most common problem that the grub ha s lost configuration and there are several way to restore it back. grub> grub> grub> grub> grub> find /boot/grub/stage1 root (hd0,1) kernel /vmlinuz root=/dev/sda2 ro initrd /initrd boot