Archive for the 'Linux / Unix' Category

Oracle Enterprise Linux 7: How to stop bash tab completion from escaping the dollar $

Gnu bash logo - By Justindorfman - Own work, CC BY-SA 4.0

Did you ever wonder why bash in Oracle Linux, Red Hat Enterprise Linux and other recent Linuxes escapes the dollar sign ($) when completing paths with the tabulator (tab) key like that:

[oracle@oratest01 ~]$ (KLM12102) ls -l \$ORACLE_BASE/diag


In bash >= 4.2.29 you have a “direxpand” option, which is NOT set by default. The bash manual explains it this way:

If set, bash replaces directory names with the results of word expansion when performing filename completion. This changes the contents of the readline editing buffer. If not set, bash attempts to preserve what the user typed.


Set this option with

shopt -s direxpand

For example in your profile or wherever you feel it is useful.

Bye and be careful, as usual
– Martin

How to disable Oracle ACFS drivers / registry resource

Sometimes an installed ACFS can cause trouble, especially if we don’t or can’t use it (e.g. when not using an UEK Linux kernel, like with SuSE Linux Enterprise Server SLES). There’s lots of documentation how to create and maintain ACFS file systems, but how to get rid of ACFS at all wasn’t so easy to guess. I had to find out how to disable Oracle ACFS drivers, because the grid infrastructure did not stop successfully when using “crsctl stop crs” or “/etc/init.d/ohasd stop”:
CRS-2799: Failed to shut down resource ‘ora.drivers.acfs’ – and thus, it was not possible to upgrade the grid infrastructure to version ( fails on first node, also when trying to stop the CRS for the same reason).

How to disable Oracle ACFS drivers - runInstaller during upgrade

Read more…

Chemnitzer Linux Tage 2015

Last weekend, I had the chance to attend the Linux Days Chemnitz. It’s an annual meeting of the German Linux family with roundabout 2,500 attendees and a FANTASTIC atmosphere. It was two days of hearing tech talks, enjoying rich nerd content and talkin’ shop.


Great atmosphere in the Great Hall


Great entrance gift: “Free Culture” by Lawrence Lessig. Thank you very much!

Read more…

New German Linux Forum (

In the last weeks, some folks were busy to build a new German Linux Forum “”, since the predecessor was systematically ruined by the commercial owners.

Especially Jean (wdp) and Hendrik (Nilpferd) invested much time and money into building the new environment. So the new forum is completely free of ads and commercials, and the content is QA’ed by a team of experienced Linux admins as moderators.

Please hang out there, and help us to (re)build a cool community.

Martin Klier (Usn)

Oracle on Linux: How to hide your password when using a wrapper script

Sometimes, a DBA has to write an externally called wrapper script for various Oracle-supplied commands accepting password inputs. A prominent and simple example is SQL*plus (sqlplus).

The Problem

The process list shows all parameters of a command that’s currently executed.

Accepts all connection infos on the command line:

 cmdstring="sqlplus ${1}/${2}@${3}"
 echo "Executed command: $cmdstring"
 sleep 999

Called by:

$ ./ system manager mydb
 Executed command: sqlplus system/manager@mydb

But now, the password is visible in the process list:

$ ps aux | grep | grep -v grep
 1000     20769  0.0  0.0  13808  1444 pts/1    S+   15:19   0:00 /bin/bash ./ system manager mydb Read more...

Oracle Clusterware issue: USM driver install actions failed (oracleoks.ko)

As I already said in my last post about “Can’t install ohasd service“, setting up Oracle Clusterware on SuSE Linux Enterprise Server (SLES) SP2 should work flawlessly, but sometimes it does not. :) This time, it was about the USM drivers.

USM driver install actions failed
/u01/app/grid/11.2.0/perl/bin/perl -I/u01/app/grid/11.2.0/perl/lib 
/u01/app/grid/11.2.0/crs/install/ execution failed

USM drivers are components (Kernel object files, extension .ko) enabling ACFS – I don’t use it on this system, but (in fact, needs a decent directory structure related to the Linux Kernel version: Again, the log file “$GRID_HOME/cfgtoollogs/crsconfig/rootcrs_<hostname>.log” was my friend: It unveiled, that the problem was somewhat related to loading oracleoks.ko. And this file is in directory “$GRID_HOME/install/usm/Novell/SLES11/x86_64/<your-kernel-version>/default/bin”. Trouble is, that good old SLES 11 SP2 has a Kernel that was not foressen by the Oracle folks implementing this piece of software.

Read more…

Oracle Clusterware fails: Can’t install ohasd service: Inappropriate ioctl for device line 5427

Setting up Oracle Clusterware on SuSE Linux Enterprise Server (SLES) SP2 should work flawlessly, but sometimes it does not. :) It turned out that this would become a pair of blog entries. Second one is about “USM driver install actions failed (oracleoks.ko)“. But step by step. On Saturday morning, failed with the following error:

Failed to install ohasd startup script, error: Can’t install ohasd service: Inappropriate IOCTL (I/O-Control) for device

Can’t install ohasd service: Inappropriate IOCTL (I/O-Control) for device at /u01/app/grid/11.2.0/crs/install/ line 5427.

/u01/app/grid/11.2.0/perl/bin/perl -I/u01/app/grid/11.2.0/perl/lib -I/u01/app/grid/11.2.0/crs/install /u01/app/grid/11.2.0/crs/install/ execution failed

There are several “My-Oracle-Support” (MOS) entries (bug notes and documents) for failing in, but not for line 5427 – and the line really matters! This script does a lot, and usually different things in different lines. :)

Whenever dealing with malfunctions, the rootcrs logfile ($GRID_HOME/cfgtoollogs/crsconfig/rootcrs_<hostname>.log) is your best friend. It appears in a not-too-verbose style, and if invokes OS- or third party commands, it quotes those outputs in a useful way – Bravo Zulu for the Oracle scripters here.

In my particular case, the problem was related to Linux’ insserv command, thats used to integrate ohasd into the SYS V startup script structure. My IBM Storage Manager Agent (service SMagent) and Oracle’s Trace File Analyzer (service init.tfa) had a dependency loop (dumbass SMagent depends on $all, /*NO COMMENT*/). In my case, I happily removed the $all dependency, and off it went.

Good luck with your GI

Oracle on AIX: How to find out the process memory usage

Calculating memory on Unix is tricky business. Especially when a complex software like Oracle Database has shared memory segments like SGA and Code Area.

One might be convinced to use the following construction to calculate the overall memory footprint of Oracle processes running on this machine:

ps -elf |egrep " oracle* | ora_.*_* " | grep -v egrep \\
| awk '{sum += $10} END {print sum/1024/1024}'

But that’s bad, since the sum is based on the SZ column of the “ps -elf” command. Unfortunately, SZ displays the full core image, but most of it is shared (remember the Oracle Code Area from the architecture diagram). So we greatly overestimate the memory footprint this way.


When you use “ps v” for a given PID, you get it more detailled: SIZE is the non shared data rump, TSIZE the shared text component of the image. In sum, they roughly add up to SZ.
(Units are all in KB)

I tried to find a solution. This is the original, overestimated version:

# ps -elf |egrep " oracle* | ora_.*_* " | grep -v egrep \\
| awk '{sum += $10} END {print sum/1024/1024}'

This one extracts the PID from “ps -ef”, executes “ps v” for each and adds them up. The greps might be a bit ugly, but it works for Oracle. :)

# for X in $(ps -ef | egrep " oracle* | ora_.*_*  " | grep -v egrep | awk '{print $2}'); \\
do ps v $X | grep ora | awk '{print $6}'; done \\
| awk '{sizesum += $1} END {print sizesum/1024/1024}'

I ran both commands on the same prod database system within the same second, so the difference should be realistic.

Stay safe

Thanks to Maxym’s old blog entry for great impressions!

Additional reading:

Speaking at COLLABORATE 14: “YOUR machine and MY database – a performing relationship!?”

I’m excited to announce that IOUG accepted my talk

“YOUR machine and MY database – a performing relationship!?”

for COLLABORATE 14 in Las Vegas.


I’d love to see you there – for tech talk, gossip and meeting old and new friends!


Databases affect machines, machines affect databases. Optimizing one is pointless without knowing the other. System administrators and database administrators will not necessarily have the same opinion – often because they know little about the opposite’s needs. This lecture was made to promote understanding – showing how the database can stress the server, and how the server can limit the database. And why two admins sometimes don’t speak the same language, not even with a developer as an interpreter.

  • Recall the different needs of different technical layers underneath a database system.
  • Understand the technical collaboration of hardware, operating system and database.
  • Plot ways how to avoid collisions, competition and concurrency.
  • Promote collaboration!

Date, time and location:

Thu, Apr 10, 2014
01:00 p.m. – 02:00 p.m.

Level 3, Lido 3003

The Venetian and Sands Expo Center
201 Sands Ave
Las Vegas, NV 89169

Presentation and papers


Martin Klier now on twitter

After ignoring the little bird telling things for quite a while, I decided to join the tweeters. Twitter might bring more color into my daily reading. :)

If you feel like, just follow me – @MartinKlierDBA

You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.