Rants & Raves - The Blog!

Astell & Kern – AK100 Review

Norman — Thu, 23 May 2013 14:38:13 +0000

Astell & Kern AK100

My wife bought me one of these because my old iRiver H340 has begun eating batteries. It’s on its third replacement now, plus I have more music than fits the H340′s hard drive. I do not buy Apple products, so any iWhatever (other than iRiver) was out of the question. Nothing personal, I just don’t like them.

Finding a suitable replacement music player with decent storage, no hard drive, and Linux compatibility (64 bit essential) was very difficult especially if you are not fond of Apple kit. (Have I mentioned that I’m not?)

The package contains the device, a nice little fabric pouch, a demo micro SD card (2 Gb) with 5 tracks on it, a screen protector for the touch screen on front and one for the back, and a charging lead. There is no charger – you can use your PC’s USB slot, your Kindle charger, your phone charger or whatever you have that is fitted with a standard USB slot to plug the lead into.

You must supply your own (decent quality) headphones – or use the optical output to feed into a quality Hi-Fi system. The headphone socket can also supply line-out levels, so can be fed directly into an amp.

The manual is in various languages, and lives on the device itself. Save space by deleting the one(s) you don’t need. Or all of them. Don’t try and print it off on A4 paper, the pages are tiny – as is the print. Use a PDF reader to view the pages.

There is a copy of “iRiver Plus 4″ for Windows on the device. Delete it. (See below!) and download the latest from iRiver.com. If you have a 32 bit Windows computer that is!

The supplied software (iRiver Plus 4) will not work on Windows 7 Professional 64 bit. It is 32 bit only. On 64 bit, it will not recognise that the device is plugged in. However, the software is not actually very good, so you are better off just using Windows Explorer and mounting the device as a USB drive. That works.

The two micro USB cards are valid up to 32 Gb but with the latest software and careful formatting, 64 Gb micro cards can be used giving a huge 160 Gb of storage. At least, that’s what I’m told, I haven’t got any 64Gb cards to try……yet!

Linux users, don’t worry. The device, and cards is recognised as USB storage. Simply drag and drop files to the (up to three) drives. Works just fine. The manual says that only Windows and Mac (32 bit) are supported by iRiver Plus 4 and even then, only the card in slot 1 is recognised. Go figure.

The supplied iRiver Plus 4 software doesn’t work under Wine on a 64 bit Linux system. It doesn’t work under Windows 64bit, so there’s not much chance of it working under Wine on a 64 bit system. 32 bit users may find that it does work under Wine. (Good Luck!)

Updating the firmware without iRiver Plus 4 is simple:

Download the zip file from http://www.iriver.com/support/download_list.asp. (Release 1.33 is available at the time of writing).
Unzip it to give you a hex file.
Copy the hex file to the root of the main drive of the device – the same location where you see Music, Manual, AlbumArt and System directories.
Unmount the USB drive that the device is pretending to be. Windows users, this means “safely remove”. Don’t just pull the plug!
It will notice the new hex file and will install it. After a power off and back on, which it does automagically, you are now running the latest release.

Make sure you have a fully charged battery though, you do not want to expire half way through the upgrade. That would make an expensive brick!

Sound quality is excellent, I’ve heard stuff on my ripped CDs that I’ve never heard before. The device plays all sorts of formats, but the best ones are FLAC, WMA and OGG. Take my advice, don’t even think about putting MP3 on this device – why would you bother, unless you really need the space savings.

The AK100 is solidly built. The chassis is a solid billet of aluminium which gives the device a decent weight. It is certainly not at all plasticy – and at the price, it shouldn’t be!

Don’t so as I did – spend ages trying to get the supplied screen protector on the front and the back case protector on the back. There is one already fitted on each side, so the ones you get in the package are spares! However, if you do do as I did, the touch screen works perfectly well through two protectors!

Controls are minimal. On the top is the power switch, input and output sockets.

The right side has only the volume knob. When playing, turning that increases or decreases the volume – but, if the screen is on, it shows the volume screen on the device and you can easily adjust the volume using the touch screen. Just drag the pointer where you want it to be.

The left side has three tiny buttons. Rewind, play, forward. Each has two functions as per the manual.

The bottom has the micro USB socket and the two slots for the micro USB cards under a heavy duty slide to open cover.

BEWARE! The manual shows only 1 micro card being inserted and shows how to align it. The bottom slot matches the diagram in the manual. However, the top slot requires the card to be inverted (turned upside down!) and put in the other way round.

So, in the bottom slot it’s as per the manual. In the top slot, turn the card over so that the contacts are showing, and put it in that way. Be very very careful, the slots are very small, and putting the cards in the wrong way round might damage things and prevent you using them.

Also, the cards have to be pushed in until they click, and the same to get them out. Use another card to do this, because it’s quite fiddly to get them in with fingers like mine. Ladies with decent nails will have no problem!

Building play lists on the device itself is interesting. You have to be playing a track to add it. It’s not impossible, just fiddly. The supplied software is apparently better to use, but as it doesn’t work on my system, I’m a tad up that creek! However, I’m a programmer and I’ll be writing my own sometime soon. Cross platform Windows, Linux and Mac (hopefully) – so drop me an email or get me on Twitter for a progress report (@NormanDunbar).

Actually, I’ve determined that the playlist is nothing more than a list of the paths to the files I want in the playlist. So, building one in a text editor is relatively simple. I’m still going to build a proper cross platform playlist builder though!

I loaded up the internal memory to almost full with a pile of FLACs, and on turning the device back on, the Auto Scan Database Rebuild told me that there wasn’t enough room to rescan the tracks to put into the database. This is interesting.

I moved a pile of directories and files onto one of the SD cards and still got the same error, however, moving a few more directories resolved the problem. I think it’s caused by the scan attempting to build the entire database in memory and then write it to the file – or something like that – and it’s unable to do it with all the tracks I have loaded. Once again, I’ll be writing something cross platform to do the needful myself.

Until I got the scan to work, I couldn’t select tracks by Album, Artists, Genre etc, only by Folder. It’s not a huge problem, my folders are arranged quite nicely by Artist then Album(s) beneath that and finally, the tracks.

I don’t do album art, and because I have the screen configured to turn off after 10 seconds anyway, I really can’t see the point of Album Art while a track is playing. I wonder if playing the same track on repeat for long enough will cause screen burn after a while? Plus, of course, it hogs valuable space for more music! Mind you, some music formats these days embed the art in the track – so if you were to purchase three tracks from Amazon, for example, you’d most likely end up with three duplicates of the same album art, embedded in your MP3s. Wonder if the art can be removed from the tracks. Hmmm.

The equaliser function is probably best left turned off. No matter what I did with it, I found the default sound settings better then anything I chose using the equaliser. It updates the sound in real time, when you stop moving your finger on the screen for a second, the sound will change. Keep moving your finger and it will patiently wait for you before changing anything sound wise.

The pouch supplied is good for protecting the device from scratches, and such like, but it is quite light and won’t take too much stress. It is certainly not suitable for attaching to a belt for portability. This device is better off locked in an inside pocket or similar. I will be on the lookout for a small sturdy case to attach this to my belt.

Talking of which, if you do want to go portable with this player, and why not, get a headphone that comes out at 90 degrees rather than a straight in plug. It makes better sense, especially if you ever bend over while wearing it around your waist. The sticky out plug gets bent over by your bending body and could knacker the socket and/or headphone plug. My Sony active noise cancelling headphones have a 90 degree plug and work great.

So, in summary:

It’s excellent, a little pricey, but it’s not Apple.
Quality of build and sound is excellent.
It handles optical in and out, if you have it.
The software is described on a Hi-Fi forum as a “steaming POS” – it is. It doesn’t work at all on 64 bit Windows systems, but to be fair, the manual does say that it must be 32 bit. It won’t run on Linux, but under Linux, you don’t actually need it.
It doesn’t play video, or allow you to read text or word files, it’s a quality music player! If you want the other stuff, buy a tablet or a smart phone!
Without the software, creating playlists is fiddly.
The database rebuild scanner cannot cope with a huge amount of files all at once.
I love it!

And finally, I used to get around 4-6 hours playback from a fully charged battery on my H340. With this AK100, I’m past that already and I still show a full charge! I’m still on the very first (5 hour) charge.

Do you think I’ve been going on too long?

As I said, because the supplied software is pants, I’ll be building my own at some point, if you are interested in a (free) copy when it’s done, contact me at the email above (assuming my email is published) which is “norman (at) dunbar-it (dot) co (dot) uk” – or @NormanDunbar on Twitter for details. But, for Windows 32 bit users – probably most of you out there – you might be ok with iRiver Plus 4.

I did have a problem with the AK100 after a few weeks of almost constant use. The volume knob on the side became intermittent. I contacted my dealer – Sound Fidelity – and they were extremely helpful and had the unit returned for investigation. When it was returned to me, I noticed that the iRiver Service Centre had updated the software to 1.33 and the knob feels a lot more substantial than before in operation.

I tried the bluetooth function on the device and it hung! I had to do a power off reset (press the ^ button on the side, and the power button on top for 7 seconds) which wiped my settings but not the music or the database. Trying bluetooth again resulted in another hang and another reset. Maybe something in 1.33 is a problem?

Time to contact iRiver support.

Cheers,
Norm.

Spatial Indexes and Oracle Errors. How to fix.

Norman — Mon, 20 May 2013 13:32:44 +0000

If, like me, you have suffered from ORA-29902 Error in executing ODCIIndexStart() routine errors where Spatial indexes are involved, the following might help you fix it.

The error involved in the following has been extracted from a log file for a system which doesn’t use Spatial or Locator itself, but calls out to a separate database which does have Locator installed. This latter database was created using Transportable Tablespaces, exported from 10.2.0.5 Enterprise Edition on HP-UX and imported into 11.2.0.3 Standard Edition on Linux x86-64.

There were a number of errors creating a few of the spatial indexes on tables, like the one that follows in the example, that had zero rows in them. Oracle Support assured us that this was not a problem. And we believed them. Sigh!

The Problem

The following query demonstrates the problem.

CONNECT CADDBA/password

SELECT * FROM TEXT_FORESHORE A
WHERE MDSYS.SDO_RELATE( A.GEOM, 
                        MDSYS.SDO_GEOMETRY(2003,81989,
                        NULL,
                        MDSYS.SDO_ELEM_INFO_ARRAY(1,1003,3),
                        MDSYS.SDO_ORDINATE_ARRAY(362000,600000,363000,601000)),
                        'MASK=ANYINTERACT QUERYTYPE=WINDOW') = 'TRUE';
SELECT * FROM TEXT_FORESHORE A
*
ERROR at line 1:
ORA-29902: error in executing ODCIIndexStart() routine
ORA-13203: failed to read USER_SDO_GEOM_METADATA view
ORA-13203: failed to read USER_SDO_GEOM_METADATA view
ORA-06512: at "MDSYS.SDO_INDEX_METHOD_10I", line 333

Working Out

I am definitely not a Spatial guru, but the above doesn’t look right to me. Looking at Google, the problem is caused by the Spatial Index being not there, missing, absent. Ok, let’s create it.

CREATE INDEX IDX_T142_GEOM ON TEXT_FORESHORE(GEOM)
INDEXTYPE IS MDSYS.SPATIAL_INDEX
PARAMETERS('TABLESPACE=CAD_PRSN_IDX_SPAT SDO_RTR_PCTFREE=0')
NOPARALLEL;

CREATE INDEX IDX_T142_GEOM ON TEXT_FORESHORE
                    *
ERROR at line 1:
ORA-00955: name is already used by an existing object

Ok, to me, that says that the index is actually present. DBA_INDEXES shows this to be the case. Apparently, it needs to be dropped and recreated, so I carry on:

DROP INDEX IDX_T142_GEOM ;
Index dropped.

CREATE INDEX IDX_T142_GEOM ON TEXT_FORESHORE(GEOM)
INDEXTYPE IS MDSYS.SPATIAL_INDEX
PARAMETERS('TABLESPACE=CAD_PRSN_IDX_SPAT SDO_RTR_PCTFREE=0')
NOPARALLEL;

CREATE INDEX IDX_T142_GEOM ON TEXT_FORESHORE
*
ERROR at line 1:
ORA-29855: error occurred in the execution of ODCIINDEXCREATE routine
ORA-13203: failed to read USER_SDO_GEOM_METADATA view
ORA-13203: failed to read USER_SDO_GEOM_METADATA view
ORA-06512: at "MDSYS.SDO_INDEX_METHOD_10I", line 10

Aha. Something different this time. Still not working though. It might be as simple as the CADDBA user not having the correct privileges. Create table and create sequence is required to create a spatial index – whether directly in the schema or as another user creating on in the schema in question. So:

CONNECT / AS SYSDBA

SELECT PRIVILEGE
FROM DBA_SYS_PRIVS
WHERE PRIVILEGE IN ('CREATE TABLE', 'CREATE SEQUENCE' )
AND GRANTEE = 'CADDBA';

PRIVILEGE
---------------
CREATE SEQUENCE
CREATE TABLE

2 rows selected.

So that’s not the problem this time. Looking into the USER_SDO_GEOM_METADATA view, for this user (every user with Spatial data should have this view) I see nothing for this table_name and column_name:

CONNECT CADDBA/password

SELECT * FROM USER_SDO_GEOM_METADATA
WHERE TABLE_NAME = 'TEXT_FORESHORE'
AND COLUMN_NAME = 'GEOM';

no rows selected

Ok, a clue. I (vaguely) know that in order to create a spatial index, that view needs some data telling it all about the column in question. As this database had been created from a legacy database (which very very rarely gets updated) I was ok to extract the data from legacy and insert it directly here.

Did I mention, each time the commands fail to create the index in question, they create the index in question? So after each failure, you have to drop it again. Sigh!

DROP INDEX CADDBA.IDX_T142_GEOM ;
Index dropped.

INSERT INTO USER_SDO_GEOM_METADATA
VALUES ('TEXT_FORESHORE','GEOM',
        mdsys.SDO_DIM_ARRAY(
             mdsys.SDO_DIM_ELEMENT('Easting', 0, 700000, .0005),
             mdsys.SDO_DIM_ELEMENT('Northing', 0, 1300000, .0005)
        ), 81989);

1 row created.

COMMIT;
Commit complete.

Now can I create the index?

CREATE INDEX IDX_T142_GEOM ON TEXT_FORESHORE(GEOM)
INDEXTYPE IS MDSYS.SPATIAL_INDEX
PARAMETERS('TABLESPACE=CAD_PRSN_IDX_SPAT SDO_RTR_PCTFREE=0')
NOPARALLEL;

Index created.

And success at long last. Spatial, I hate you! Does the query work now?

SELECT * FROM TEXT_FORESHORE A
WHERE MDSYS.SDO_RELATE( A.GEOM, 
                        MDSYS.SDO_GEOMETRY(2003,81989,
                        NULL,
                        MDSYS.SDO_ELEM_INFO_ARRAY(1,1003,3),
                        MDSYS.SDO_ORDINATE_ARRAY(362000,600000,363000,601000)),
                        'MASK=ANYINTERACT QUERYTYPE=WINDOW') = 'TRUE';

no rows selected

After all that work, no rows selected is exactly the correct answer. The table is empty, so I would have been very surprised to see anything other than that response.

The Solution

The solution to my particular problem was to:

Drop the so called missing index.
Make sure correct data is in USER_SDO_GEOM_METADATA for the table and column in question. Each user with Spatial data will have one of these views, so you need to be in the appropriate user.
Create the index again.
Test the failing query, and it should work.

Cheers.

Need to Compile a 32 bit Application on a 64 bit Linux System?

Norman — Mon, 13 May 2013 19:27:03 +0000

I have a small utility that I need to use from time to time. I wrote it back in 2009 when 32 bit systems were all the rage. It reads a certain type of data file and outputs lots of, ahem, interesting information about the contents. When I compiled it on my 64 bit laptop, it produced complete and utter rubbish! How did I fix it?

The data file that the utility reads is a Firebird database file which has a number of long fields in various structs. On a 32 bit Linux system, a long is the same size as an int – both are 32 bit.

Thanks to Vlad Khorsun, one of the Firebird Development team (of volunteers) I realised that under a 64 bit system, ints remained as 32 bit while longs got a wee bit bigger at 64 bits – hence my structs were complete nonsense under a 64 bit system.

There are two solutions, well, at least two:

Rewrite the code to take account of the world of 32 and 64 bit development systems. This would involve discovering, somehow, if the utility was being built on a 32 or 64 bit system, and changing the definition of a long variable or struct member accordingly.
Compile the system and make it believe it’s running on a 32 bit system. No code changes required!

Being lazy, and because there’s only a slim possibility of this code being released into the wild, I chose the latter option. It turned out to be quite simple.

First, I needed to install the gcc multilib support packages on my Mint KDE system:

sudo apt-get install gcc-multilib

Now, when I compile the code, I use the -m32 option to gcc, as follows:

gcc -m32 -o fbdump32 fbdump.c

Simple, and it works. However, the correct solution would be to correctly define my structs with proper C/C++ standard data types that guarantee that the width will always be 32 bits and/or determine whether I’m compiling on 32 or 64 bit systems. For example:

// Data types that guarantee specific widths:

[u]int8_t are 8 bit.
[u]int16_t are 16 bit.
[u]int32_t are 32 bit.
[u]int64_t are 64 bit.


// Maximum and minimum for these are defined as:

[U]INT8_MIN and [U]INT8_MAX
[U]INT16_MIN and [U]INT16_MAX
[U]INT32_MIN and [U]INT32_MAX
[U]INT64_MIN and [U]INT64_MAX


// Am I compiling on a 64 bit system by any chance? 
#if __WORDSIZE==64
... // Do 64 bit stuff here.
#else
... // Do 32 bit stuff here.
#endif

Cheers.

This Listener Problem is Driving Me Mad!

Norman — Fri, 10 May 2013 16:39:28 +0000

I have been looking at this far too long, and I’m stumped. I resolved a similar problem yesterday on another server. That was down to the ORACLE_HOME setting in listener.ora having a ’1′ in it rather than a ’2′. Took ages to spot that.

Anyway, here the stuff you’ll need to know to sort this for me, or suggest stuff. It’s a question on Oracle L seeing as there is a lot of evidence to post.

As ever, server names etc have been changed to protect the innocent!

Update We have a solution! Scroll to the bottom for details.

Oracle and OS Versions

Oracle Database: Standard Edition, 11.2.0.3 64 bit.
Server: SLES 10 sp 4
Uname -r: 2.6.16.60-0.97.1-smp
hostname: orcl11gserver

The Problem

In a word, setting ORACLE_SID and connecting to a user/password works fine. Connecting to user/password@alias gives the following error:

ERROR:
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
Process ID: 0
Session ID: 0 Serial number: 0

Database Info

I can connect to the database, both as sysdba and as a non-sysdba user provided I don’t use the listener:

$ sqlplus / as sysdba
...
Connected.

SQL> show parameter listener

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
listener_networks                    string
local_listener                       string
remote_listener                      string


SQL> show parameter db_name

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_name                              string      orcl11g


SQL> show parameter service

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
service_names                        string      orcl11g.world


SQL> select * from global_name;

GLOBAL_NAME
--------------------------------------------------------------------------------
orcl11g.WORLD

Listener.ora

lsnr_orcl11g =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS_LIST =
        (ADDRESS = (PROTOCOL = TCP)(HOST = orcl11gserver)(PORT = 1521))
      )
    )
  )

SID_LIST_lsnr_orcl11g =
  (SID_LIST =
    (SID_DESC =
      (GLOBAL_DBNAME = orcl11g)
      (ORACLE_HOME = /opt/oracle/product/11.2.0.3/db_1)
      (SID_NAME = orcl11g)
    )
)

DYNAMIC_REGISTRATION_lsnr_orcl11g = off
SUBSCRIBE_FOR_NODE_DOWN_EVENT_lsnr_orcl11g=OFF

Tnsnames.ora

orcl11g,orcl11g.world =
  (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(Host = orcl11gserver)(Port = 1521))
    )
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = orcl11g)
    )
  )

Sqlnet.ora

NAMES.DIRECTORY_PATH= (LDAP, TNSNAMES, EZCONNECT, HOSTNAME)

Oratab

orcl11g:/opt/oracle/product/11.2.0.3/db_1/:N

Tnsping

$ tnsping orcl11g:

Used parameter files:
/opt/oracle/product/11.2.0.3/db_1/network/admin/sqlnet.ora


Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS_LIST = (ADDRESS = (PROTOCOL = TCP)(Host = orcl11gserver)(Port = 1521))) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = orcl11g)))
OK (0 msec)

Listener Status

$ lsnrctl status lsnr_orcl11g
...
TNSLSNR for Linux: Version 11.2.0.3.0 - Production
System parameter file is /opt/oracle/product/11.2.0.3/db_1/network/admin/listener.ora
Log messages written to /opt/oracle/diag/tnslsnr/orcl11gserver/lsnr_orcl11g/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=orcl11gserver.testds.ntnl)(PORT=1521)))

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=orcl11gserver)(PORT=1521)))
STATUS of the LISTENER
------------------------
Alias                     lsnr_orcl11g
Version                   TNSLSNR for Linux: Version 11.2.0.3.0 - Production
Start Date                10-MAY-2013 16:51:52
Uptime                    0 days 0 hr. 0 min. 0 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /opt/oracle/product/11.2.0.3/db_1/network/admin/listener.ora
Listener Log File         /opt/oracle/diag/tnslsnr/orcl11gserver/lsnr_orcl11g/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=orcl11gserver.testds.ntnl)(PORT=1521)))
Services Summary...
Service "orcl11g" has 1 instance(s).
  Instance "orcl11g", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully

Where is Listener Running From?

ps -ef|grep -i ls[n]r_cds

oracle   21180     1  0 16:51 ?        00:00:00 /opt/oracle/product/11.2.0.3/db_1/bin/tnslsnr lsnr_orcl11g -inherit

Listener Log

The listener log shows the connection attempt being made, and established ok witha result code of zero.


 10-MAY-2013 17:25:56 * (CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=orcl11g)(CID=(PROGRAM=sqlplus)(HOST=orcl11gserver)(USER=oracle))) * (ADDRESS=(PROTOCOL=tcp)(HOST=10.57.18.116)(PORT=12633)) * establish * orcl11g * 0

Client Trace

Don’t worry, I’m not about to paste an entire ADMIN level trace here. But looking in one, I saw this extract:

nsbasic_brc:type=12, plen=11
nsbasic_brc:what=17, tot =11
nsbasic_brc:packet dump
nsbasic_brc:00 0B 00 00 0C 00 00 00  |........|
nsbasic_brc:01 00 01                 |...     |
nsbasic_brc:exit: oln=0, dln=1, tot=11, rc=0
nioqrc: found a break marker...
nioqrc: Recieve: returning error: 3111

This is sort of interesting, as it seems to indicate I got a break from somewhere or something! I saw this on my other similar problem as well, so it’s the same in the two trace files, but I solved the other problem by correcting the Oracle Home in listener.ora. Not this time!

The Solution

There are many people on oracle-l who took the time to look at the problem, so thanks to all. There are, however, two people to whom I am extremely grateful. They took mere minutes to discover what had been staring me in the face all day, and the winners are:

@martinberx on Twitter.
David Barbour on oracle-l.

Both noticed that in /etc/oratab, the Oracle Home path had a trailing slash, while in the listener.ora, it did not. Sheesh!

Thanks to both.

The Fix

The fix was relatively simple:

With the current (wrong) oratab settings in force, shut down the database and the listeners. (The problem affected a number of databases/listeners on this server, not just the one I used in the above example.)
Edit oratab to remove the trailing slash.
Restart the listeners and databases with the new improved oratab.
Test – it all “just works”.

ORA-03262 While Dropping Empty Datafile

Norman — Thu, 02 May 2013 14:37:28 +0000

Tired of trying to drop empty datafiles from a tablespace, which you know is empty? Keep getting errors telling you the data file isn’t empty? Getting frustrated with the whole thing? Me too. This link has the reason and solution.

Internet Explorer Won’t Upload Files to MOS?

Norman — Thu, 02 May 2013 08:55:18 +0000

Are you forced to use Internet Explorer at work? Are you, like me, forced to use an old, insecure, broken version of IE at work, because it’s the Government Standard version? And are you, like me, unable to upload evidence files to My Oracle Support?

You need to go to Tools, then Internet Options.
On the Security tab, click the Custom Level button.
Now find and enable the Include Local Directory Path option.
OK your way back out, restart IE, and Robert is your mother’s brother.

This works fine and has been tested on IE7, IE8, IE9 and possibly (but untested) IE10. Other browsers are not affected, but I’m told that Chrome also has problems.

Firefox, on the other hand, just works.

Hope this helps, unlike me, you may be able to find and click the above options. Our (Government) security policies have that option disabled and I’m not allowed to change it. I’m not allowed to use Firefox either. Go figure.

Security – sometimes it’s there to stop you doing your job.

Cheers.

Statspack Snapshot Fails ORA-01400 Cannot Insert NULL …

Norman — Wed, 01 May 2013 10:13:45 +0000

Oh hum. An 11.2.0.3 Enterprise Edition production database has statspack taking a regular snapshot under the control of a dbms_scheduler job. For no apparent reason, the snapshot started failing with ORA-01400 Cannot insert NULL into PERFSTAT.STATS$SYSTEM_EVENT.EVENT. This was an interesting one to fix.

The following is the investigative process, in brief.

Test the snapshot process with a manual one – same error.
Google and My Oracle Support aka MOS, were no help whatsoever. I was on my own! Twitter was useful and Noons (@wizofoz2k) suggested a statspack mismatch could cause this error.
Knowing that I had reinstalled statspack on this database a few weeks ago led me to drop and reinstall statspack and to recreate the jobs required to take regular snapshots and to purge old data. No joy, same problem.
Hunt down the code in V$SQL to see what’s going on here. A quick script helped out:
```
SELECT sql_fulltext
FROM   v$sql
WHERE  DBMS_LOB.INSTR(sql_fulltext, 'SYSTEM_EVENT') <> 0
AND    DBMS_LOB.INSTR(sql_fulltext, 'INSERT') <> 0;
```
That showed an insert statement, as expected, reading the EVENT column from V$SYSTEM_EVENT – which, given half a brain, makes sense! I didn’t have half a brain at the time – as will become obvious!
Another quick script showed that there were 5 rows in V$SYSTEM_EVENT that were NULL:
```
SELECT count(*) 
FROM sql_fulltext
WHERE event IS NULL;

COUNT(*)
--------
       5
```
WTF?

Looking at the EVENT column, showed a huge load of crud and nothing much like a proper Oracle event. Some of the data were:

rwp err: No dash in error string
r removing error %d is [ ][ ]
rrupt, error stack is [ ][ ]
down and process is starting up
indicate must count [ ][ ]
...

WTF? (The sequel!)

The next stop – I did say I didn’t have half a brain didn’t I – was the alert log, where I should have been looking in the first place! Bingo!:
```
WARNING: Oracle executable binary mismatch detected.
Binary of new process does not match binary which started instance.
```

And there we have it. Noons was correct in as much as the version of statspack in use – 11.2.0.3 EE – didn’t match the running database binary which was 11.2.0.3 SE. Somehow, someone (no, not me – but thanks for asking!) had managed to start the database running on SE rather than EE.

I’m thinking that this could have been done when an SE environment was enabled in a session, and someone simply did an export ORACLE_SID=whatever not realising that EE was required. This posting might help in that case!

After shutting down the database, making sure that the correct environment was set, a restart of the database got rid of the messages in the alert log, and a snapshot was successfully executed.

So, an interesting challenge that could have been resolved earlier if I’d gone straight to the alert log rather than dicking about thinking I knew that it must have been the reinstall I did previously! That’ll teach me to think then!

And by the way, I know (oops) from previous experience, that the snapshot code in V$SQL will have table names and commands in upper case, which is why I used upper case tests for INSERT and SYSTEM_EVENT in the script above. If I wasn’t so sure, I’d have done this instead:

SELECT sql_fulltext
FROM   v$sql
WHERE  DBMS_LOB.INSTR(upper(sql_fulltext), 'SYSTEM_EVENT') <> 0
AND    DBMS_LOB.INSTR(upper(sql_fulltext), 'INSERT') <> 0;

Cheers.

My Step-Father’s Cremation

Norman — Thu, 25 Apr 2013 15:33:22 +0000

Antonio Enrico Martini was cremated on 19th April 2013. He had been a chef for many years and one of his friends, whose name I shall add here when I remember it – read this Address to a chef when Enrico retired in 1987. The version below, was amended from the 1987 version. Both are based on Address to a Pudding by Rabbie Burns.

Address To A Chef

Fair fa' your honest sonsie face,
Great Enrico o' th' Italian Race,
Ye cam' amang us lang years since, an' did enthrall,
Wi' savoury dish o' string an' mince, cried Spaggi Bol!

Mony's the nicht we've supped your fare
Chosen frae menu wi lovin' care;
Aft times tempted tae hae some mair, o' Tagliatelli.
The end result, ye can be share – an ower-full belly!

Aye, greed's a tempter, there's nae doot,
When faced wi' grub that's unco' good,
Tae get stuck in tae favoured food, like Canelloni,
An' then consume, in reckless mood, Zabaglione!

That menu's fu' o' names exotto,
Like Lasagne and Risotto,
Try them a'! - That's been my motto – have nae fear,
This chef, say I, (no voce sotto), has nae peer.

Now every king must hae his queen,
Wha better then than Bonnie Jean!
Her work, it seems, is nivver deen, an' man she's fleet,
The quickest thing ye've ivver seen on jist twa feet!

Well, that wis then – his course is run,
The time has come; it waits for none,
Sae fare-thee-well tae San Remo's son, oor heart's wi' Jeannie,
Rest aye in peace the only one full-proof Martini.

As Amended 19th April 2013

Rest in peace Enrico. Your Lasagne will always be the best.

Swap 2 Values, in SQL, Without Using a Temporary Variable

Norman — Wed, 24 Apr 2013 11:44:28 +0000

Recently, I saw a mention of an interview question for SQL developers. It was something along the lines of:

There is a table with a sex column. It has been discovered that the values are swapped around and need to be corrected. How would you swap all ‘M’ values to ‘F’ and all ‘F’ values to ‘M’, while leaving the other values untouched, in one single SQL statement and without requiring the use of any temporary variables.

So, how would you do it? Here’s my example.

First, the before look:

select sex, forename from bedrock;

S  FORENAME
-  --------
M  Wilma
   Dino
M  Betty
F  Fred
F  Barney
U  Pebbles

6 rows selected.

I think we can safely say that the sex column is somewhat back to front. My solution was to use the CASE statement in an SQL UPDATE command:

update bedrock
set sex =
  case sex 
    when 'F' then 'M'
    when 'M' then 'F'
  end;

Finally, the after look:

select sex, forename from bedrock;

S  FORENAME
-  --------
F  Wilma
   Dino
F  Betty
M  Fred
M  Barney
U  Pebbles

6 rows selected.

It appears to be safe to commit!

Using the CASE statement is useful. In the old days, we would need something like the following:

Update bedrock set sex = 'T' where sex = 'M';
Update bedrock set sex = 'M' where sex = 'F';
Update bedrock set sex = 'F' where sex = 'T';

For huge tables, that could have taken a while, and it’s highly unlikely that a sex column would be indexed, so three full table scans would have been the order of the day.

Using CASE we get away with a single scan, plus, the statement short circuits when it hits the first matching WHEN clauses. So, if the first sex value was ‘F’ it would be changed to ‘M’ however it would then stop checking and would not change the newly set ‘M’ back to an ‘F’.

Ever deleted a Data File From a Running Database?

Norman — Tue, 23 Apr 2013 11:07:08 +0000

http://jarneil.wordpress.com/2013/04/23/recovering-from-rm-rf-on-a-datafile/ has all you need to know to help avert a total disaster. Tanel Poder, who knows these things, advises that the file should be quiesced or made read only before attempting the recovery.

Firefox Running Extremely Slowly?

Norman — Fri, 05 Apr 2013 13:50:03 +0000

Firefox starts off ok, but soon starts running slower and slower, until it eventually starts to time out on connecting to some pages. The error messages is “the server took too long to respond” however, Firefox might be telling porkies.

If you attempt to access the same URL in Opera, Chrome or, if you must, Internet Explorer, you may find that it is responding quite happily and speedily, while another try in Firefox takes ages to connect or fail again.

You need to sort out a couple of options:

Edit->Preferences (on Linux) or Tools->Options (I think, on Windows)
Click Advanced
Click General tab
Uncheck Use hardware acceleration when available
Click Network tab
Click Settings button beside Configure how Firefox connects to the internet
Check No proxy but be aware that this option might be restricted if you use Firefox at work.
Click OK button
Click Close button

You might need to restart Firefox after this, I didn’t on Linux, but try a link without restarting first and see what happens. On my system, it works perfectly quickly again.

TNS-01189: The listener could not authenticate the user

Norman — Tue, 19 Mar 2013 14:31:53 +0000

Ever see this error? I have, just today. An interesting one to debug. I got there in the end though.

The database is running on a two node VERITAS cluster. To protect the innocent, I shall refer to these as node_04 and node_05, for that is similar to their real names! The database is not RAC, it runs on one node or the other, but never both. There is one instance and one database.

The DNS and/or whatever they use to ensure that traffic goes to the correct node (I have a good grasp of the technicalities, haven’t I?) uses a host name of sgxxxx where xxxx is the database name. Tnsnames.ora and listener.ora use this special hostname in the configuration for the database alias resolution, and the listener.

The database is running on node_04 right now, but can be forced to run on node_05 due to a failure or a manual fail over. The listener follows the database and runs on the same node – wherever that happens to be. It is not possible for the listener to run on node_04 with the database on node_05, and vice versa.

I was attempting to get a status from lsnrctl for the listener:

Node_04> lsnrctl status lsnr_xxxx

LSNRCTL for Linux: Version 11.2.0.2.0 - Production on 19-MAR-2014 10:11:03

Copyright (c) 1991, 2010, Oracle. All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=sgxxxx)(PORT=1603)))
TNS-01189: The listener could not authenticate the user

It failed, as shown above. Interesting because it has been working “forever”.

Running a tnsping on the database had no problems. It returned with a zero millisecond response time. Pretty quick. So we know the listener is up and running, and responding, we just can’t get a status.

Node_04> tnsping xxxx

TNS Ping Utility for Linux: Version 11.2.0.2.0 - Production on 19-MAR-2014 10:12:23

Copyright (c) 1997, 2010, Oracle. All rights reserved.


Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = sgxxxx)(PORT = 1603)(CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = xxxx)))
OK (0 msec)

Next step was to check the listener.ora file to see if there was a password set for the listener – there was not. Hmmm.

Insert a delay here while various obvious stuff was checked, and found to be ok. Everything works unless we use the listener to access the database. At that point, the TNS error occurs again.

Eventually, I wondered what the sgxxx “special” hostname resolved to:

Node_04> ssh oracle@sgxxxx
password: 

(lots of login information, disclaimers, legal warnings etc here)

Node_05> exit
Node_04>

Did you notice? I did, straight away. The special host name resolves to the other node in the VERITAS cluster. It should be resolving to the same node as the database – node_04.

The solution is to get the network guys and gals to fix it. It’s a cluster of two nodes and should work accordingly. Sgxxxx needs to resolve to the cluster node where the database is running. It wasn’t.

Nasty, but fun to debug.

Setting Oracle Environment in Scripts

Norman — Mon, 18 Mar 2013 09:56:52 +0000

A quickie! How do you set the correct Oracle environment in scripts? Do you hard code? You better not.

I’ve lost count of the times I’ve ended up with, for example, a 10g database running with bits of the 9i software hanging around. It leads to monumental problems that can be hard to track down.

Moral: Do not hard code Oracle environment details.

This is what I usually do:

#!/usr/bin/env bash

export ORAENV_ASK=NO
export ORACLE_SID=my_sid
. oraenv
export ORAENV_ASK=YES

# Rest of bash script goes here....

Reagrdless of how many times the database version gets an update, you will still always have the correct Oracle software on the path and in the environment when the script runs.

Today, I learned a new way to do the above, with much less typing. You can see an example here.

#!/usr/bin/env bash

. oraenv <
All those years of Oracle and I never figured out that I could user a “here” document. Thanks Gokhan.

Linux Command to Retrieve Hardware Serial Numbers etc

Norman — Mon, 18 Mar 2013 09:34:40 +0000

Ever needed to obtain the serial number (or other details) for a remote server? Couldn’t be bothered to walk/run/drive/fly all the way there just to read a sticky label on the back or bottom of said server? Read on then.

The command you want to run, as root, is dmidecode. For example, to get the make and model and serial number of a server, do this:

dmidecode -t system

The result will be similar to:

# dmidecode 2.11
SMBIOS 2.5 present.

Handle 0x0002, DMI type 1, 27 bytes
System Information
        Manufacturer: Dell Inc.
        Product Name: Vostro 1720
        Version: Null
        Serial Number: 996C4L1
        UUID: Not Settable
        Wake-up Type: Power Switch
        SKU Number: Null
        Family: Vostro

Handle 0x000F, DMI type 12, 5 bytes
System Configuration Options
        Option 1: Jumper settings can be described here.

Handle 0x0018, DMI type 32, 20 bytes
System Boot Information
        Status: No errors detected

Other options for the -t parameter are:

bios – tells you all about your bios.
system – tells you about the system hardware.
baseboard – all about the mother board.
chassis – all you need to know about the “box” the system is made up of.
processor – fairly obvious.
memory – again, fairly obvious.
cache – information about your CPU cache.
connector – what sockets are present on the computer. USB, firewire, ethernet etc.
slot – appears to be the bus information, and voltages present, supplied etc.

There’s brief help available:

dmidecode --help

Usage: dmidecode [OPTIONS]
Options are:
 -d, --dev-mem FILE     Read memory from device FILE (default: /dev/mem)
 -h, --help             Display this help text and exit
 -q, --quiet            Less verbose output
 -s, --string KEYWORD   Only display the value of the given DMI string
 -t, --type TYPE        Only display the entries of given type
 -u, --dump             Do not decode the entries
     --dump-bin FILE    Dump the DMI data to a binary file
     --from-dump FILE   Read the DMI data from a binary file
 -V, --version          Display the version and exit

However, to find out the different types you can supply, you need to supply an erroneous type:

dmidecode -t left_leg

Invalid type keyword: left_leg
Valid type keywords are:
  bios
  system
  baseboard
  chassis
  processor
  memory
  cache
  connector
  slot

I’ve just used the command to obtain information about a server located 150 odd miles away from my comfy chair, running in an unattended site. That saved me a bit of time!

Have fun.

Bee Tragedy

Norman — Sun, 03 Mar 2013 15:52:43 +0000

Yesterday, Saturday 2nd March 2013, I went to check on my two bee hives. They were fine and healthy with plenty feed stock last time I checked.

Yesterday, nothing. Every single bee was dead. There was plenty food, so it seems that the recent damp foggy nasty cold weather must have either killed or severely weakened them.

I have no bees.

Two dead bee hives, ready for sterilisation.

This is all that remains, two dead hives. I need to clean and sterilise before I even attempt to house any new bees here. You can never be too careful.

I have checked the hives and bodies over, but I saw no signs of disease.

Here be Dragons!

Norman — Mon, 25 Feb 2013 12:33:55 +0000

I took this photo in Scotland, along the Crinan Canal, back in March 2009. Anyone know what it is? Click the image for a full screen version. (Warning – it’s 4000px by 3000px and 5.6 Mb file-size. The original file-size is even larger – it’s almost 12Mb!)

Update: it’s a Southern Hawker. Thanks Howard!

Scottish Dragonfly. Crinan Canal, March 2009.

Transportable Tablespace Migrations with Expdp/Impdp

Norman — Thu, 21 Feb 2013 15:25:15 +0000

In the old days of exp/imp doing a Transportable Tablespace export/import was relatively simple – unless you had Spatial data, in which case, it wasn’t. Then we got hold of expdp/impdp and it became “different“. It now seems that in order to do a Transportable Tablespace import with impdp, you don’t tell it to do one! Confused?

In the old days, you told both exp and imp which tablespaces you were transporting using the TRANSPORT_TABLESPACE and TABLESPACES parameters to exp, and the same with imp. Life was good and symmetrical back then!

With expdp, we have a similar parameter whereby we simply list the tablespaces we wish to transport in the TRANSPORT_TABLESPACES (note, plural) parameter.

You would think that a similar arrangement would exist with impdp wouldn’t you? Well it does. Sort of!

There is indeed a TRANSPORT_TABLESPACES (note, plural) parameter on impdp but, if you use it, you must also specify a database link name in the NETWORK_LINK parameter and that link must exist in the importing database and it must point back to the exporting database. There is no intermediate dump file, the metadata is unloaded from the source database over the database link. Smart?

Not quite. Imagine that you have a production database to export and then import into a QA database, for example. However, the two are on separate networks, and there’s an air gap between the two. You cannot set up a database link from the QA database to the Production one to get at the metadata for the Transportable Tablespace import.

If you try, without a database link in place and working, impdp will simply barf.

The solution is hidden away in a note attached to the TRANSPORT_TABLESPACES in the Utilities manual, impdp section, in the locked filing cabinet, in the office in the basement, behind the leopard guarding the stairs, down the dark corridor etc. (Douglas Adams.)

What you have to do is not use the TRANSPORT_TABLESPACES parameter as you would imagine, instead, you list the transported data files using the TRANSPORT_DATAFILES and avoid the TRANSPORT_TABLESPACES like the plague! Goodby symmetry, it was nice of you to drop in!

So far, so bad. However, my original problem with deferred segment creation still exists even with impdp instead of imp.

$ impdp directory=... dumpfile=... logfile=... transport_datafiles=file1.dbf ...
...
Processing object type TRANSPORTABLE_EXPORT/TABLE
ORA-39083: Object type TABLE:"NORMAN"."TEST_TABLE" failed to create with error:
ORA-01647: tablespace 'TEST_TABLESPACE' is read-only, cannot allocate space in it
...

Of course it’s read-only, it doesn’t exist in the database because impdp is supposed to create it, load up the metadata and connect that with the data files etc. Then I have to login and make the tablespace read write when I’m done.

Sigh!

Rsync – To Slash or Not To Slash?

Norman — Mon, 11 Feb 2013 20:31:37 +0000

A quickie! Rsync is great for making sure that a destination directory is synchronised with a source directory. However, do you add a slash to the source and/or destination directory names, or do you not?

The answer is, it depends. Without a slash on the source directory means copy both the source directory, and the contents (recursively if specified) to the destination directory while adding a trailing slash means only copy the contents of the source directory, recursively if specified, to the destination. Easy?

If we take the following as the source directory:

$ tree testing

testing
|-- another
|   |-- wilma
|-- betty
|-- fred
|-- nested
    |-- barney

The destination is an empty directory named test_backup.

No Slashes

The first test has no slashes on any of the directories.

$ rm -r test_backup/*
$ rsync --archive --recursive testing test_backup
$ tree test_backup

test_backup
|--testing
   |-- another
   |   |-- wilma
   |-- betty
   |-- fred
   |-- nested
       |-- barney

You can see that the whole hierarchy of the testing directory has been recreated within the destination directory.

Slash on Destination Only

The second test, after clearing out the destination directory, adds a slash to the end of the destination directory.

$ rm -r test_backup/*
$ rsync --archive --recursive testing test_backup/
$ tree test_backup

test_backup
|--testing
   |-- another
   |   |-- wilma
   |-- betty
   |-- fred
   |-- nested
       |-- barney

So, there’s no difference there. A slash on the destination directory appears to have no effect.

Slash on Source Only

$ rm -r test_backup/*
$ rsync --archive --recursive testing/ test_backup
$ tree test_backup

test_backup
|-- another
|   |-- wilma
|-- betty
|-- fred
|-- nested
    |-- barney

This is different. The contents of the source directory have been duplicated into the destination directory.

Slashes on Both

$ rm -r test_backup/*
$ rsync --archive --recursive testing/ test_backup/
$ tree test_backup

test_backup
|-- another
|   |-- wilma
|-- betty
|-- fred
|-- nested
    |-- barney

And this one again shows that having a slash on the destination directory has no effect.

Setting up SSH on Western Digital My Book World Devices

Norman — Mon, 11 Feb 2013 16:43:36 +0000

I have a network attached 1 Tb hard drive. It’s a white book, Western Digital “My Book World” device. I use it for backups. I needed to set it up to allow the backup scripts ssh access without a password. Here’s how I did it.

The web interface was used in the normal way to create a user named my_backups.
A share was set up automatically for this new user. The share name is the same as the user name – my_backups.
using advanced mode->network, I ensured that NFS was enabled and that only devices on my internal network were able to access the shares. This is not necesary for the backups though, but it allows me to NFS mount the share as required.
I logged onto the WD device using ssh, as root. I needed to edit the /etc/passwd file to move the home directory for my new user from /shares to /shares/my_backups otherwise, setttng up ssh affects (potentially) all users. And besides, I want my users to have their own home, not a shared one.
The default permissions on the my_backups share is to allow group writes. Ssh will not work if this is the case, so that has to go, as root:
```
# chmod g-w /shares/my_backups
```
The ownership of the my_backups share is also dubious. So another quick change is called for, as root:
```
# chown my_backups /shares/my_backups
```
Now I can login using ssh as the my_backups user, so logout as root, and back in again as my_backups.

Now set up all the stuff (technical term!) required by ssh:

$ cd
$ pwd
/shares/my_backups

$ mkdir .ssh backups
$ chmod 600 .ssh
$ ls -al
...
drw-------    2 my_backu jewab           6 Feb 11 15:56 .ssh
drwx------    2 my_backu jewab           6 Feb 11 16:27 backups

$ cd .ssh
$ touch authorized_keys
$ chmod 700 authorized_keys
$ ls -l
-rwx------    1 my_backu jewab           0 Feb 11 15:58 authorized_keys

$exit

Now, I copy my ssh public key to the new authorized_keys file, from my laptop, which is the device I’m backing up up to the shared drive:
```
$ cd ~/.ssh
$ cat id_dsa.pub | ssh root@wd 'cat >> ~my_backups/.ssh/authorized_keys'
root@wd's password:
```
You will notice that I’m connecting as root to do this. For some reason, on the device, attempting to do this as my_backups gives me an error sh: cannot create .ssh/authorized_keys: Permission denied which is strange, as the file exists and is correctly owned by the my_backups user. However, there are a few foibles in the OS that is running on the device, so logging in as root works!
Now I can login as my_backups without a password, from my laptop. And more to the point, my backup script can do it too.
```
ssh my_backups@wd


~ $ pwd
/shares/my_backups
```
Talking of backup scripts, it’s nothing more than an rsync command. The rsync option --archive cannot be used as it won’t let the --times part of it run and lots of errors are scrolled up the screen.
To get around this, I’m using --perms --delete --update --verbose --links --progress --recursive instead. The destination on the rsync command is specified as my_backups@wd:backups which does the job nicely!

Ahem, of course, the first time I did this, I neglected to create the backups directory didn’t I? So what happened on the first actual run? Rsync happily --deleted the .ssh directory holding my keys, so further testing didn’t work and requested the password that I had worked so hard to get rid of!

Cannot Send Emails, or Read Web Servers From Oracle 11g

Norman — Tue, 05 Feb 2013 15:50:43 +0000

Accessing a web server or an email server, directly from within a database, used to be quite simple. However, it all stops working at 11g. Why is that and what can be done to fix it?

Introduction

Prior to Oracle 11g, any user in the database wishing to use the various network packages – UTL_HTTP, UTL_SMTP, UTL_TCP, UTL_MAIL etc, and their predecessors, only requires to be granted EXECUTE privileges on the appropriate package(s).

From 11g onwards this is no longer the case. Execute privilege is still required, however, further fine grained access control has been added to the database to restrict the networks reachable, port ranges allowed, and even down to the actual start and end dates and times that the access will be allowed.

This fine grained control is achieved via ACLs. (Access Control Lists.) and the XML Database (XDB) product which must be installed on the database.

Information Requirements

If an application wishes to use network resources, then the following information must be obtained, in advance, of the application being set up in the database. It is advised that it be documented in the database and/or application documentation

Email

For an application connecting directly to a mail server, not via sendmail, or equivalent:

Hostname of the email server, or IP address of same.
Port number(s) in use for the email server.
Database username (ie schema names) of all schemas that will be executing the code that sends emails.
If required, a range of dates and times when the service should be accessible.

HTTP

For an application connecting directly to a web server or URL:

Hostname of the web server, or IP address of same.
Port number(s) in use for the email server.
Database username (ie schema names) of all schemas that will be executing the code that connects to the web or email server.
If required, a range of dates and times when the service must be accessible.

Setting up ACLs

The SYSDBA, or a user with DBA role granted, must execute the following code to create a new ACL:

BEGIN
  DBMS_NETWORK_ACL_ADMIN.create_acl (
    Acl => 'email_http_access.xml',
    Description => 'Allows access to UTL_HTTP, UTL_SMTP etc',
    Principal => 'USERNAME',
    Is_grant => TRUE,
    Privilege => 'connect',
    Start_date => SYSTIMESTAMP,
    End_date => NULL);

  Commit;
End;
/

The parameters, and points to note are:

ACL – is the name of an xml file. ACLs are kept in the XML Database product, so XDB must be installed. The data is kept in the table XDB.XDB$ACL and also in a folder on the database server.
Description – is a meaningful, please, description of what the ACL is created to allow.
Principal – the main user account in the database which requires access to the network utilities. This may be a role or a user. The parameter is case sensitive. Beware.
Is_grant – true grants the privilege, false denies the privilege.
Privilege – use ‘connect’ for UTL_TCP, UTL_SMTP, UT:_MAIL and UTL_HTTP. Use ‘resolve’ for UTL_INADDR. Beware, this parameter is also case sensitive.
Start_date and End_date are null by default. Set these to a particular TIMESTAMP to prevent the ACL from being active until or after the specific date given.
The commit is mandatory.

Adding Users to ACLs

Additional users and or roles are added to the ACL using the add_privilege procedure. The parameters are as above with the omission of the description parameter and the addition of the position parameter.

BEGIN
  DBMS_NETWORK_ACL_ADMIN.add_privilege (
    Acl => 'email_http_access.xml',
    Principal => 'OTHER_USERNAME',
    Is_grant => TRUE,
    Privilege => 'connect',
    Position => NULL,
    Start_date => NULL,
    End_date => NULL);

  Commit;
End;
/

The parameters and points to note are:

Position – defines the position in the list of ACL privileges, for this principal. If an ACL higher up the list denies access and one lower down grants it again, the latter takes precedence.
The commit is mandatory.

Assigning ACLs to a Network Resource

Once created, the ACL must be assigned to a network using the assign_acl procedure. This is where the principal(s) which have been granted privileges in an ACL, are given access to a network resource for the duration of the ACL.

BEGIN
  DBMS_NETWORK_ACL_ADMIN.assign_acl (
    Acl => 'email_http_access.xml',
    Host => 'IP_or_hostname',
    Lower_port => 80,
    Upper_port => 80);

  DBMS_NETWORK_ACL_ADMIN.assign_acl (
    Acl => 'email_http_access.xml',
    Host => 'IP_or_hostname',
    Lower_port => 443,
    Upper_port => 443);

  Commit;
End;
/

The parameters and points to note are:

ACL – is the name of an existing ACL xml file.
Host – defines the network resource allowed. Only those network resources mentioned here will be allowed to be accessed. Wild cards can be used, but are advised against. See below.
Lower_port and upper_port define the port range permitted. For default HTTP servers this will be ports 80 and 443 (for HTTPS access). For email servers, the default SMTP port is 25.
Only the specific host(s) and port(s) are accessible via the ACL. If you allow access to a web server on port 80 only, then attempting to use https on port 443 will be rejected.
The commit is mandatory.

Network Resources

A network can be specified as follows:

A hostname – ‘oracle.com’ or ‘capgemini.com’. In this case, only that hostname will be accessible from the database. If the IP address changes, it will continue working once DNS has propagated.
An IP address – ’84.37.86.172′. Again, only this network address will be accessible.
An IP range – ’84.37.86.*’. All devices on the 84.37.86 network are accessible. The ‘*’ indicates “everything” and can be used in one or more of the ‘dotted quads’.
Everything – ‘*’ – best avoided on the grounds that someone could use your database to set up and execute attacks on any device anywhere in the world. Use the principal of least privilege when setting up these ACLs.

Ports

By omitting the upper and lower ports, everything on the host is accessible.
If you specify a range of port numbers then all ports in that range, inclusive, are accessible.
If you need to set up a discontinuous port range, you must call the assign_acl procedure once for each range, as per the example above – ports 80 and 443 only are accessible.

A Worked Example

The following example shows the user norman attempting to access an HTTP web site directly to pull down a script, all from within SQL*Plus.

Initial Test

The URL shown below is fictitious. The web site does not actually exist. The process shown below, however, does work and I’m grateful to Tanel Poder for the details, which you can find here.

SQL> connect norman/secret
Connected.

SQL> set lines 1000
SQL> set trimspool on
SQL> set trimout on
SQL> set pagesize 0
SQL> set long 99999999 longchunksize 99999999
SQL> set feedback off
SQL> set head off
SQL>
SQL> spool useful_script.sql
SQL>
SQL> select httpuritype('http://fictitious.oracle.com/useful_script.sql').getCLOB()
  2> from dual;

ERROR:
ORA-29273: HTTP request failed.
ORA-06512: at "SYS.UTL_HTTP", line 1819
ORA-24247: network access denied by access control list (ACL)
ORA-06512: at "SYS.HTTPURITYPE", line 34

As you can see, it failed due to ACL reasons.

Setup

The following script is executed by the SYSDBA:

BEGIN
  DBMS_NETWORK_ACL_ADMIN.create_acl (
    Acl => 'email_http_access.xml',
    Description => 'Allows access to UTL_HTTP, UTL_SMTP etc',
    Principal => 'NORMAN',
    Is_grant => TRUE,
    Privilege => 'connect',
    Start_date => SYSTIMESTAMP,
    End_date => NULL);

  Commit;
End;
/

BEGIN
  DBMS_NETWORK_ACL_ADMIN.assign_acl (
    Acl => 'email_http_access.xml',
    Host => 'fictitious.oracle.com',
    Lower_port => 80,	-- Default web server port
    Upper_port => 80);	-- Ditto

  Commit;
End;
/

Successful Test

SQL> connect norman/secret
Connected.

SQL> set lines 1000
SQL> -- etc etc. See above.

SQL> spool useful_script.sql
SQL>
SQL> select httpuritype('http://fictitious.oracle.com/useful_script.sql').getCLOB()
  2> from dual;

-- Lots of output scrolls up the screen here ...

SQL> spool off

Further Details

Further details are available from Tim Hall’s web site where he takes the Oracle docs and makes sense of them. Most of the indforation given above is blatently borrowed from Tim’s excellent explanation.

I had to use the above today to allow a database procedure access to an email server. The application is being ported from 10g to 11g and for some unknown reason, emails started failing! Tim’s blog post was extremely helpful in explaining and helping resolve the problem.

This blog post is simply a reminder to myself as to what had to be done to fix it. I’ll need it again soon I expect! For the whole story, get over to Tim’s web site. Don’t stick around here, trust me, it’s not worth it!

Thanks Tim.

Update 1st March 2013: There’s an interesting blog post here where a database trigger is used to create the ACLs after a particular user is granted execute on UTL_MAIL. You might find it interesting.

ORA$AT_SA_SPC_SY Jobs failing?

Norman — Mon, 28 Jan 2013 15:18:07 +0000

Oracle has raised an alert in the alert.log and created a trace file as well, for a failed DBMS_SCHEDULER job with a strange name which doesn’t appear in DBA_SCHEDULER_JOBS or DBA_SCHEDULER_PROGRAMS – what’s going on?

An extract from the alert log and/or the trace file mentioned in the alert log shows something like:

*** SERVICE NAME:(SYS.USERS) ...
*** MODULE NAME:(DBMS_SCHEDULER) ...
*** ACTION NAME:(ORA$AT_SA_SPC_SY_nnn) ...

Where ‘nnn’ in the action name is a number.

No matter how hard you scan the DBA_SCHEDULER_% views, you will not find anything with this name. What is actually failing?

Oracle 11.1.0.6 onwards stopped listing these internal jobs in DBA_SCHEDULER_JOBS, as they did in 10g, and instead lists them in DBA_AUTOTASK_% views. However, not by actual name, so don’t go looking for a TASK_NAME that matches the above action name. You will fail.

There are three different autotask types:

Space advisor
Optimiser stats collection
SQl tuning advisor

The tasks that run for these autotask ‘clients’ are named as follows:

ORA$AT_SA_SPC_SY_nnn for Space advisor tasks
ORA$AT_OS_OPT_SY_nnn for Optimiser stats collection tasks
ORA$AT_SQ_SQL_SW_nnn for Space advisor tasks

See MOS notes 756734.1, 755838.1, 466920.1 and Bug 12343947 for details. The first of these has the most relevant and useful information.

UPDATE: My original failing autotask has been diagnosed by Oracle Support as bug 13840704 for which a patch exists here for 11.2.0.2 and 11.2.0.3.

Oracle document id 13840704.8 has details, but it involves LOBs based on a user defined type. In this case, Spatial data in an MDSYS.SDO_GEOMETRY column.

The view DBA_AUTOTASK_CLIENT won’t show you anything about a specific task, with the above names, but will show you details of what the overall ‘client’ is, There are three:

select client_name, status
from dba_autotask_client;

CLIENT_NAME                     STATUS
------------------------------- --------
auto optimizer stats collection ENABLED
auto space advisor              ENABLED
sql tuning advisor              DISABLED

I can see from the task name in the alert log and trace file, that my failing task is a space advisor one, so, by looking into the DBA_AUTOTASK_JOB_HISTORY view, I can see what’s been happening:

select distinct client_name, window_name, job_status, job_info
from dba_autotask_job_history
where job_status <> 'SUCCEEDED'
order by 1,2;

CLIENT_NAME        WINDOW_NAME     JOB_STATUS JOB_INFO
------------------ --------------- ---------- -------------------------------------------
auto space advisor SATURDAY WINDOW FAILED     ORA-6502: PL/SQL: numeric or value error...
auto space advisor SUNDAY WINDOW   FAILED     ORA-6502: PL/SQL: numeric or value error...

So, in my own example, the auto space advisor appears to have failed on Saturday and Sunday. Given that this is an internal task, and nothing I can do will let me know about the invalid number problem, I need to log an SR with Oracle on the matter. However, as I don’t want my fellow DBAs to be paged in the wee small hours for a known problem, I have disabled the space advisor task as follows:

BEGIN
  dbms_auto_task_admin.disable(
    client_name => 'auto space advisor',
    operation   => NULL,
    window_name => NULL);
END;
/

PL/SQL procedure successfully completed

Checking DBA_AUTOTASK_CLIENT again, shows that it is indeed disabled:

select client_name, status
from dba_autotask_client
where client_name = 'auto space advisor';

CLIENT_NAME                     STATUS
------------------------------- --------
auto space advisor              DISABLED

Enabling it again after Oracle Support have helped resolve the problem is as simple as calling dbms_auto_task_admin.enable with exactly the same parameters as for the disable call:

BEGIN
  dbms_auto_task_admin.enable(
    client_name => 'auto space advisor',
    operation   => NULL,
    window_name => NULL);
END;
/

PL/SQL procedure successfully completed

When enabling and/or disabling auto tasks, you must use the CLIENT_NAME as found in DBA_AUTOTASK_CLIENT view.

The full list of DBA_AUTOTASK_% views is:

DBA_AUTOTASK_CLIENT
DBA_AUTOTASK_CLIENT_HISTORY
DBA_AUTOTASK_CLIENT_JOB
DBA_AUTOTASK_JOB_HISTORY
DBA_AUTOTASK_OPERATION
DBA_AUTOTASK_SCHEDULE
DBA_AUTOTASK_TASK
DBA_AUTOTASK_WINDOW_CLIENTS
DBA_AUTOTASK_WINDOW_HISTORY

Hope this helps!

Installing Oracle Java 7 on Linux Mint 13

Norman — Fri, 11 Jan 2013 15:49:51 +0000

I use Java only when I have to, and only ever the JRE (Java Runtime Environment) – there is no way I’ll use Java for development work. I’d rather eat my own ear wax to be honest!

Linux Mint 13 comes with OpenJDK installed, and a system I use which runs the fop FO Processor, barfs with a Java.Lang.NullPointer Exception if OpenJDK is used.

The following is brief instructions on how to install Oracle’s java on Linux Mint.

Download Java JRE

Go to here and click on the “Free Java Download” button.

Click on Linux or Linux-x64 depending on whether you are running a 32 or a 64 bit Linux system. When prompted, select a suitable place to save the file, and click OK.

Remeber, you don’t want the various Linux*.rpm packages – they are for Red Hat/Fedora/Scientific Linux/Centos/OpenSuse/Oracle Linux distributions.

When the download is complete, you should have a saved copy of something like jre-7u10-linux-x64.tar.gz

Install Java

In a shell session (don’t be afraid!), change to the directory where you saved the downloaded file. In my case, that was Downloads/Java in my home directory. Then uncompress the file.

$ cd ~/Downloads/Java
$ tar -xvzf jre-7u10-linux-x64.tar.gz

Lots of filenames will whizz past on the screen. Wait for the process to finish.

You now need to be root, so, use sudo sh or su - or whatever to get you into a root session. I use su as my root user has been given a secure password.

In a root session, you need to move all the files you just extracted to the /usr/lib/jvm

$ su -
password:

$ cd ~norman/Downloads/Java
$ ls
jre1.7.0_10  jre-7u10-linux-x64.tar.gz

$ mv jre1.7.0_10 /usr/lib/jvm/

That’s all done now, Java is in the correct place. All that remains is to configure the system to use it in preference to anything else that is present.

Still working as root, run the update-alternatives command, as follows, to tell the system about the new Java version, and to set it as default.

$ update-alternatives --install /usr/bin/java java /usr/lib/jvm/jre1.7.0_10/bin/java 1065

update-alternatives: using /usr/lib/jvm/jre1.7.0_10/bin/java to provide /usr/bin/java (java) in auto mode.

And also the following:

$ update-alternatives --install /usr/bin/javaws javaws /usr/lib/jvm/jre1.7.0_10/bin/javaws 1065

update-alternatives: using /usr/lib/jvm/jre1.7.0_10/bin/javaws to provide /usr/bin/javaws (javaws) in auto mode.

That’s it, Oracle’s Java is now installed and will be used as the default.

If, for some strange reason, I had installed a full development kit, rather than just the runtime, I would have required the following two commands to be run as well – to configure the default java compiler and jar tool.

$ update-alternatives --install /usr/bin/javac javac /usr/lib/jvm/jre1.7.0_10/bin/javac 1065
$ update-alternatives --install /usr/bin/jar jar /usr/lib/jvm/jre1.7.0_10/bin/jar 1065

Changing the Default Java Version

The above makes my development tool happy as fop no longer barfs with an exception and my FO source code is happily converted into pdf documents.

What happens when some other program doesn’t like to play with Oracle’s Java and needs the original OpenJDK version instead?

Simple, change the default again:

update-alternatives --config java
There are 3 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status
------------------------------------------------------------
* 0            /usr/lib/jvm/jre1.7.0_10/bin/java                1065      auto mode
  1            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java   1061      manual mode
  2            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1051      manual mode
  3            /usr/lib/jvm/jre1.7.0_10/bin/java                1065      manual mode

Press enter to keep the current choice[*], or type selection number:

The update-alternatives displays all known installed versions of Java and lets you choose the one you want. The first entry is the current one, and you simply type in the number of the one you want instead.

Have fun. (If there’s such a thing as fun where Java is involved!)

Travel Sucks!

Norman — Sun, 30 Dec 2012 14:56:27 +0000

WARNING – If you are easily offended, I would advise that you do not read any further. You have been advised, proceed at your own risk and don’t bother to complain, because if you do, I will most likely delete your comments. I have the power!

As I write this, the temperature outside is minus 60C – minus 76F in old money, or American degrees – but then again, I am about 36,000 feet up and still have three hours to go until Manchester.

I am heading home, after two weeks in Tenerife over Christmas 2012. It was my Mother in Law’s idea to go away for Christmas, and she’s been a right royal PITA for a fortnight! Suffice to say I will not be holidaying with her ever again.

Other than the wittering one, the usual crop of “chavs” and other people’s effing kids are driving me to drink, yet again.

Why is it that so many people think it is acceptable to force about three hundred other people to listen to their whining brat for four and a half hours flying time? Not to mention the three hours of “hanging about in bloody airport” time as well.

There is a reason we have kennels you know, next time, leave the noisy little bastard there if you absolutely must have a holiday. (Alternatively, do the environment a huge favour and stop breeding!)

They say travel broadens the mind. I say “Bullshit!”

What travel actually does is remind you of how nice it is to stay at home, with the doors locked, and something decent (with David Attenborough or Brian Cox in it) on TV.

Travel is also a great reminder of just how widespread the UK’s import of the American obesity problem actually is.

If I am allowed only 20Kg luggage allowance and have to pay an extortionate fee for each ounce over my allowance, how come some fat, over-eating, self inflicted – and most likely diabetic – person, who probably weighs three times what I do, gets charged the same price per seat as I do? What about their “excess baggage” charges?

Especially when they think it’s ok for their spare blubber to spill over into my seat? And “do you mind if I raise the seat arm between us so that I can get a bit more comfy?”

Actually, I do effing mind. I paid for a seat on this plane and so did you. You did not pay half my fare so you are not entitled to half my seat space as well as your own. Two words – the second one is “off” and the first rhymes with “truck”! Catch my drift?

Don’t like my attitude? Guess what? I don’t like your blubber spilling over and touching me! You deal with your problem and you might find me a bit more agreeable in future. In fact, you will find me more agreeable, because normally I am a considerate, courteous person – but you have seriously pissed me off!

And smokers, yes you! Your drug habit is your own affair, please keep it that way.

Please do not shut yourself in a telephone box and chain smoke your way through your last 200 because you can’t last all the way home. You stink! You may not know it, but you are tremendously obnoxious to non-smokers as well as other smokers. It’s absolutely disgusting. I may puke very soon! (Or spew, as we Jocks normally say!)

The in flight movie on this trip appears to be something called “Miles To Go”. It is crap.

It has been on since we left – almost two hours ago – and so far, no dialogue. In fact, there has been nothing but what appears to be a satellite view of a small aeroplane, similar to this one, heading roughly North towards Manchester.

I’ve saw something very similar once before when flying to and from, Australia. It was a crap movie then and this latest version has not improved things at all.

(That last bit was supposed to be amusing – it may not have worked!)

I have luckily got extra leg room seats. Well, I say luckily, I paid extra for a seat that can accomodate someone over 4’6″ without their knees being somewhere close to their forehead for the duration of the flight.

Surely a plane should have sufficient leg room in every seat?

Mind you, I have to say that the standard cattle class seat on this Thomson Airways Boeing 737-800, has far more leg room than any Monarch flight I’ve ever been on. For an excellent rant on that subject, search this blog for “Monarch”.

Oh great! The obnoxious brat two seats away has now started playing with one of his bloody “musical” toys. Everything he does is now accompanied by plinky-plonky “music” and the seemingly obligitory screaming and high pitched squealing so prevalent amongst children these days.

The “right on” and so called “modern” parents think that whispering “ssh” at the screaming little bar steward is helping. Well they must be deaf, because the little darling is paying them no heed, and continues to wake the other sleeping passengers. People actually want these things? What the fuck for?

Have sex for fun, and stuff the “multiplying” bit so loved by the Roman Catholic Church!

Two more effing hours to go. What with the human whale next to me and that damned kid not far (enough) away – like on another planet – I could be in for a long prison sentence any time soon! I might be out in 15 years or so, if I behave myself.

As for entertainment, there is none. The “in seat” entertainment centre ( a place to plug in your specially purchased headset) has nine channels of total silence, and one of what sounds to me to be a 440 Hz sinusoidal tone. Could be an A but wtf do I know about music? The only thing I play is CDs and the odd parp on my didjeridoo!

The tone sound is, unfortunately, not loud enough to drown (now there’s a thought!) out the din from the wonder child two seats away. I hate him with a vengeance! Not just because of his incredibly ill mannered behaviour, but because the little shit was sat in the same bloody seats on the way out and screamed all the fucking way there as well!

According to the “movie”, we are no longer over the sea and have just made landfall over the Spanish coast, somewhere near, or North, of Vigo.

Did I mention the in flight “meal”? No? I won’t bother then, other than over priced, and tasteless!

The human whale did manage two meals, one after the other, then wiped his hands on the seat in front. Disgusting or what? Not as disgusting as his metronomic farting – fart fart fart fart – every two minutes. Jesus!

Oh excellent! Now we have a “wonderful opportunity” to buy assorted “stuff” from the “duty free” trolley. I think I shall give it a wide berth.

Noise cancelling headphones are great on flights. They really do cancel out most of the din from the airflow and engines. At least my Sony ones do.

Unfortunately, they don’t appear to do anything about screaming kids. Now there’s a market, ready made for a set of headphones that do! I would be first in the queue for those at any price!

I did mention being driven to drink a while back, near the start of this rant. The brandy is not helping much.

I’ve drunk the whole 5Cl “bottle” (which is of course priced similar to a half bottle in normal life) and the screaming brat is no easier to bear.

Over the North coast of Spain now, heading up towards Biscay, and they have decided to add in some turbulence. That isn’t helping the brat, nor my typing!

Travel sucks! And I hate it. However, my wife loves it and the threat of divorce finds me stuck on yet another stressful commercial flight. (If I could afford my own plane, say a Honda Jet, I would travel more often but driving myself!)

Another child has just wandered past. He too is sitting two seats away. The difference is, he is quiet, well behaved and a credit to his parents. I can almost see the appeal – but there’s no way I will ever, ever, have a child!

Heading towards Brest in France now, some way to go, but for some reason, the turbulence has stopped and so has the screaming child from hell. How long can the peace and quiet possibly last I wonder?

As it happens, oops – shades of Jimmy Savile there – not long. The demon son of Satan’s cat has started squealing again. 30 seconds was the answer.

And now the battery in my noise cancelling headphones is dwindling and the cancellation is fading in and then cutting out. How positively irritating! I wonder how much worse this flight can get.

An hour to go. Is it worth trying to sleep? You never know, sleep might calm me down. They have just dimmed the cabin lights. Let’s see if it works. Wish me luck.

Happy New Year – if it comes! Here’s to a peaceful, child free 2013.

RMAN Active Database Clone – Different Servers, Same Structure

Norman — Thu, 06 Dec 2012 18:18:30 +0000

This post is all about cloning an 11g database from one server to another using an RMAN active database clone. This is not being done for Standby Database purposes, only to duplicate an existing database onto another server.

The physical structure on both servers is the same, some path names have been changed.

Source Structure

Database SID: msmdppr
/srv/msmdp/oradata/msmdppr/
/srv/msmdp/flashback_area/msmdppr/

Everything hangs off /srv/msmdp/oradata/msmdppr and there are data, index, temp, undo, redo, ctrl, dbs, diag directories there.

The redo and ctrl directories are also to be found under the flashback_area for this database. A copy of the redo files are held there and there are some of the control files as well. This is a separate LUN from the oradata structure.

These are all part of a multi-spindle SAM, so it’s not as bad as it looks having everything in the “same place”.

Destination Structure

Database SID: msmpr
/srv/msm/oradata/msmpr/all as above
/srv/msmdp/flashback_area/msmpr/all as above

The destination structure will end up the same as the source, and only the names will change. Where the source has /msmdp/ or /msmdppr/ the destination will be /msm/ or /msmpr/

Destination Preparation

Build the desired directory structures for the new database. In this example, I did the following, as the oracle user:

$ mkdir -p /srv/msm/oradata/msmpr/
$ mkdir -p /srv/msm/flashback_area/msmpr/

$ cd /srv/msm/oradata/msmpr/
$ mkdir -p data index temp undo redo ctrl dbs changetracking
$ mkdir -p diag/rdbms/msmpr/msmpr/adump

$ cd /srv/msm/flashback_area/msmpr/
$ mkdir redo ctrl

Create a temporary password file for the new database. Set the password to be the same as the source database. It won’t work otherwise. Ask me how I know?

Source Preparation

Make sure when you start the source database that there are no warnings about any deprecated parameters in the spfile. If there are, check the alert log and remove them. If you don’t, they will get transferred to the destination database and the clone will fail!
Recreate a pfile from the spfile. You must have an up to date pfile. You can use a minimum pfile on the destination server, but I find it is better to be explicit. I’ve had problems.
Copy the new pfile over to the destination server. You can put it straight into $ORACLE_HOME/dbs if you wish, or put it somewhere else, and create a symbolic link to $ORACLE_HOME/dbs. The latter is what I did. All individual database startup and password files live in the /dbs/ directory and are linked from there.

Further Preparation

The remainder of the work needs some parts to be done on the source server and others to be done on the destination server.

On the destination server, edit the pfile that was created on the source server and copied over. You need to change all occurrences of the source database name to the destination one, and make sure that all your paths etc are correctly defined for the new database
Now add the following two parameters to the pfile:
- db_file_name_convert=’/srv/msmdp/oradata/msmdppr/’,
  ‘/srv/msm/oradata/msmpr/’,
  ‘/srv/msm/flashback_area/msmpr/’,
  ‘/srv/msm/flashback_area/msmpr/’
- log_file_name_convert=’/srv/msmdp/oradata/msmdppr/’,
  ‘/srv/msm/oradata/msmpr/’,
  ‘/srv/msm/flashback_area/msmpr/’,
  ‘/srv/msm/flashback_area/msmpr/’
They will convert the filenames to the destination format. Beware, I have found that these parameters only substiture at the start of a parameter, not where the parameter includes the text to be replaced.
Startup the destination database in NOMOUNT mode.
On the destination server, create a listener for this database and start it up.
On the destination server, create a tnsnames.ora entry for the new database.
On the destination server, test that you can connect to the destination database with sqlplus sys/password@msmpr as sysdba. If it works, carry on, otherwise, fix it.
On the source server, create a tnsnames.ora entry for the destination database.
On the source server, test the connection. You need to be able to connect to the database with sqlplus sys/password@msmpr as sysdba
On the source server, if the database is running in NOARCHIVELOG mode, shut it down, then restart it in MOUNT mode. If the database is running in ARCHIVELOG mode, it can either be OPEN or MOUNTed.
On the source server, set the environment for the source database (msmdppr) and start RMAN. You may need to specify $ORACLE_HOME/bin/rman to get the correct “rman” if you are on a Linux server. There are other rmans!
In the RMAN session, run the following commands:
```
connect target sys/password
connect auxiliary sys/password@msmpr

duplicate target database to msmpr
from active database
nofilenamecheck
password file;
```
then leave RMAN to do its thing.

In my case that meant an overnight job as there was around 6.5Tb to clone. Luckily RMAN, when configured for performance, will skip the completely virgin blocks in a data file which makes things a little quicker.

The RMAN commands are saying to clone the source database and copy accross the password file from the source database to the destination one. If you have any users with SYSOPER or SYSDBA roles granted, their passwords will be in the password file, so you’ll need it on the destination server as well.

When it finishes, you will find the password file and a new spfile for the destination database in $ORACLE_HOME/dbs. if necessary, move them to where you want them, and create symbolic links back to $ORACLE_HOME/dbs.

When Things Go Wrong

Problems happen. I’ve had my share, especially with RMAN clones!

In the event that the clone exercise fails at some point, don’t panic. Log out of RMAN on the source server, and go back in.

On the destination server, shut down the database, you can ABORT if necessary. Delete the spfile for the new database, if it still exists, then restart the destination database in NOMOUNT mode again.

Fix the problem, and start again from the connect target ... command in RMAN.

One thing I did notice, in my xxxx_file_name_convert parameters, I set it to convert ‘/msmdp/’,'/msm/’,'/msmdppr/’,'/msmpr/’ and it ignored the second pair of substitutions. It seems as if it can cope with the first pair being part of the file names in question, but not the second pair, they remained unconverted and my data file copies failed because the /msmdppr/ directory wasn’t to be found.

in the end I gave up on it completely, and just forced it to convert the entire thing.

Good luck, RMAN, when it’s working is great. When you struggle with it, you struggle!

It’s Not a PiBow, It’s a Raspberry PiBoard!

Norman — Mon, 03 Dec 2012 15:17:22 +0000

Having a Raspberry Pi is fun, but like the days of my ZX-81, Spectrum and QL (I see a Sinclair habit there!) you get all sorts of bits hanging off and making it all look untidy. Here’s how to fit a whole pile of “stuff” into the space of an A4 sheet of paper.

I know it’s the size of an A4 sheet because that’s what I drew it on before cutting the wood. The image shows the final result. (Click it for a bigger picture.)

The finished PiBoard, everything fits into a space the same size as an A4 sheet of paper.

From the top left, working clockwise, we have the following:

A 60Gb USB hard drive.
Sitting on the drive, is a 400 pin breadboard currently stuffed full of Adafruit Pi Cobbler breakout connector.
A 4 way powered USB hub. This powers the Pi, the USB drive, the WiFi connector and the wireless keyboard and mouse dongle.
My Pi. It lives in a PiBow case. The GPIO ribbon cable works the Cobbler breakout or the Gertboard. There is just enough room to change the SD card or to remove it for backups. The sound and the composite video outputs are both accessible. The white box on a cable poking out the back is an HDMI to VGA converter box.
My Gertboard. I was lucky enough to get one of the first ones that had to be built from a large kit of parts. I had much fun putting that together, and even more fun when it all “just” worked!
The pear shaped “thing” in the middle of the PiBoard, partially hidden under the GPIO ribbon cable, is a USB extension, supplied with my wireless router, to plug the WiFi dongle into. You can just make out the dongle standing upright on top of the extension.

The board iktself is a half inch, sorry, 12 mm thick piece of MDF which I cut to size and sanded smooth while wearing a face mask to avoid the dust, which can be nasty stuff.

Evenrthings is stuck down with double sided sticky foam that was left over from when the new kitchen was installed (I knew it would come in handy one day!) and, in the case of the Gertboard, it’s also screwed down.

Underneath, to avoid scratching whatever I place it on, the board has a wide strip of single sided sticky foam, from the same kitchen left overs, down each side to act as non-slip feet.

In use, I plug it it and switch on. It either runs headless or I can connect it to a VGA cable to an old 12″ LED telly with a VGA input. It can be run on WiFi or cabled Ethernet depending on whatever else I have plugged in at the time!

All this makes moving the Raspberry Pi around the house quite simple. With a quick edit to config.txt, I can have it running on the 22″ HDMI monitor, or another edit and it’s back to the small VGA telly. Most of the time, it just sits there, running headless.

Raspberry Pi’s are putting the fun back in computing.

Where Does OraInventory Live? Can I Move it?

Norman — Fri, 30 Nov 2012 16:07:06 +0000

Looking for the location of oraInventory on a server? Want to know where it is? Read on.

There is a file, known to Oracle, which holds the location of the inventory. Of course, it isn’t in the same place on every server, but the ones I know of have it as follows:

Linux: /etc/oraInst.loc
HP-UX: /var/opt/oracle/oraInst.loc
Windows: Registry at HKLM/software/oracle/inst_loc

For any other Unix, you can find it (as the oracle or root user) with:

find / -type f -name oraInst.loc -print

Once you know where it lives, you can move it simply. The following example moves it from the current location, found in oraInst.loc to /opt/oracle/oraInventory:

$ cat /etc/oraInst.loc

inventory_loc=/u01/app/oracle/oraInventory
inst_group=oinstall

$ cd /opt/oracle
$ cp -Rp /u01/app/oracle/oraInventory ./

$ vi /etc/oraInst.loc
  :1
  s?/u01/app?/opt?
  :wq

$ cat /etc/oraInst.loc

inventory_loc=/opt/oracle/oraInventory
inst_group=oinstall

Job done! Although it might be wise to take a backup of the original location, just in case, and then delete it from the old location:

$ cd /opt/oracle
$ tar -cvzf u01.app.oracle.tgz /u01/app/oracle
...

$tar -tzf u01.app.oracle.tgz   ## Just checking ...
...

$cd /u01/app/
$pwd                           ## Safety check!
/u01/app

$ls                            ## Another safety check!
oracle

$rm -Rf oracle                 ## Getting nervous yet? I am!

And that it, all done.

Using “chopt” to Enable and Disable Oracle Options

Norman — Fri, 23 Nov 2012 15:52:15 +0000

As you may know, Oracle databases come with a number of options. Some of these cost extra and if inadvertantly installed, Oracle must be paid money – note, you don’t have to be using them, only have them installed, to require payment. So what do you do if you need to remove an option?

In the old days, you used to have to rebuild the oracle binaries to add or remove options. To do this you needed to know the names of a number of make targets – not for the faint hearted.

From 11g onwards, the process is much simpler. Oracle now supply the chopt (change option) utility to make your DBS’s life simple.

The utility is supplied with Enterprise and Standard Editions. It allows you to enable or disable the following options:

Data Mining
Database Vault
Oracle Label Security
OLAP
Partitioning
Real Application Testing

You can run the utility with no parameters to see what it does and how you should call it in anger:

$ chopt

usage:

chopt

So, for example, to disable partitioning because the junior DBA has mistakenly installed it (all these options are selected and installed by default in Enterprise Edition!) then all you do is set the correct Oracle Home using oraenv in the normal manner, then:

chopt disable partitioning as the following example demonstrates:

$ chopt disable partitioning

Writing to /srv/oracle/product/11gR1/db/install/disable_partitioning.log...
/usr/bin/make -f /srv/oracle/product/11gR1/db/rdbms/lib/ins_rdbms.mk part_off ORACLE_HOME=/srv/oracle/product/11gR1/db
/usr/bin/make -f /srv/oracle/product/11gR1/db/rdbms/lib/ins_rdbms.mk ioracle ORACLE_HOME=/srv/oracle/product/11gR1/db

You can check the log file named on the first line of output for details. Enabling an option is just as simple:

$ chopt enable partitioning

Writing to /srv/oracle/product/11gR1/db/install/enable_partitioning.log...
/usr/bin/make -f /srv/oracle/product/11gR1/db/rdbms/lib/ins_rdbms.mk part_on ORACLE_HOME=/srv/oracle/product/11gR1/db
/usr/bin/make -f /srv/oracle/product/11gR1/db/rdbms/lib/ins_rdbms.mk ioracle ORACLE_HOME=/srv/oracle/product/11gR1/db

You can only enable or disable a single option at a time, unlike when you are running the old style make commands where you could specify to turn them all on or off in one go. Progress?

How To Resize a VirtualBox Hard Drive

Norman — Wed, 14 Nov 2012 18:01:23 +0000

Have you ever found that the virtual hard drive attached to one of your VirtualBox VMs is getting too small? Want to know how to make it beigger?

It’s simple. The steps are as follows:

Disconnect the drive from your VM. The VM should be shut down and not snapshotted for this step. You can either use the VMs storage settings to disconnect the hard drive, or go to File->Virtual Media Manager, choose the correct hard drive, and click the release button.
In a command line session, type the following command – you do not have to be root:
```
VBoxManage modifyhd /path/to/vdi/file.vdi --resize nnnn
```
The size given, nnnn in the above, is the new size in Mb and must be bigger than the present size. You will then see something like the following example where I resize a virtual hard disc to 12Gb (12288 Mb):
```
norman@hubble $ VBoxManage modifyhd /data/VirtualBox/HardDisks/Antix.vdi --resize 12288
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
```
Go back to the VM’s storage settings, and re-attach the newly resized hard drive.
Start the VM.
Run the disc partitioner tool of your distro to extend an existing partition into the new free space, or to create a new partition. You may add the extra space to an existing logical volume if you are using LVM devices.

As mentioned above, you cannot reduce the size of a virtual disc. If you try, you will see the following:

norman@hubble $ VBoxManage modifyhd /data/VirtualBox/HardDisks/Antix.vdi --resize 6144
0%...
Progress state: VBOX_E_NOT_SUPPORTED
VBoxManage: error: Resize hard disk operation for this format is not implemented yet!

That’s all there is to it!

Oracle’s Deferred Segment Allocation Breaks Transportable Tablespace Imports.

Norman — Wed, 14 Nov 2012 14:04:38 +0000

In order to downgrade an 11.2.0.3 Enterprise Edition database to Standard Edition, I had to use a Transportable Tablespace export/import. Because the default setting for DEFERRED_SEGMENT_CREATION is TRUE, the tablespace import barfed with numerous “IMP-00017: following statements failed with ORACLE error 1647:” errors. Want to know why?

The ORA-01649 error is “Tablespace is read only, cannnot allocate space in it” which is interesting as I’m imprting a Transportable Tablespace dump file and all the tablespaces are read only after being created, until I manually make then read write.

In the source database, the “broken” tables all have zero rows in them, and have no entry in DBA_SEGMENTS which means that the Deferred Segment Allocation feature has indeed done its stuff, and deferred allocating a segment until the first row of data is entered (but not necessarily committed!) into the table.

Note. The DEFERRED_SEGMENT_CREATION is also defaulted to TRUE in Standard Edition databases, but the parameter has no effect in these, as the following shows:
Connected to:
Oracle Database 11g Release 11.2.0.3.0 - 64bit Production

SQL> show parameter deferred_seg

NAME                       TYPE     VALUE
-------------------------- -------- -------------
deferred_segment_creation  boolean  TRUE

SQL> create table test(a number);
Table created.

SQL> select table_name, segment_created
  2  from user_tables
  3  where table_name = 'TEST';

TABLE_NAME           SEG
-------------------- ---
TEST                 YES
So, even with an empty table, in Standard Edition, the deferred segment allocation does not take place. This is exactly why I’m having problems running a transportable tablespace import from an Enterprise Edition database to a Standard Edition one. It seems that Standard expects everything to have at least one allocated segment.

If the importing database is Enterprise Edition, this problem doesn’t occur.

The quick workaround is to run a table based export, using exp or expdp, of the affected tables, and import that at the receiving database using imp or impdp.

The longer term workaround is to make sure that DEFERRED_SEGMENT_CREATION is set to FALSE in the spfile.

In the meantime, I’m logging a bug with Oracle as the DBMS_TTS.TRANSPORT_SET_CHECK procedure should identify these tables and warn about them, or, the import should correctly import them anyway.

Backup & Check a Raspberry Pi SD Card

Norman — Mon, 22 Oct 2012 18:56:26 +0000

Ever wanted to backup your Raspberry Pi’s SD card, but didn’t know who to ask? Me too. Read on ….

The first thing to remember is that you should really always have a backup of your SD card. In theory every time you make a change, but in practice, it will be less frequently than that!

I have had two cards corrupt themselves when I managed to lock my Pi completely, and the only recourse was to pull the power. Unfortunately, in both cases, I was left with a card that the Pi could read the first partition from, but then couldn’t mount the second, so no operating system.

Luckily I had a backup, but it was of a 4Gb SD card and I’m running on 8 Gb, so I hjad to restore and then extend the second partition to use the remaining 4 Gb of unallocated space. Normally I’d restore the 8 Gb image to an 8 Gb card.

These instructions are for Linux only. If you use one of the Windows utilities for writing to your SD card, there should be an option to read from it and write to a file, if so, this is what you need to use. Beware, there may need to be two reads if you have two partitions, with some of these utilities.

Make a Backup

Plug your card into your computer, no, not your raspberry Pi , and wait a second or two for it to settle down.

If you have a card slot, you most likely have the card on /dev/mmcblk0 and if you have it in a USB adaptor of some kind, it’s most likely going to be /dev/sdx where ‘x’ is the digit representing the highest device of this kind.

To find out for certain, do the following, either using sudo, or as root:

fdisk -l
...
...
Disk /dev/mmcblk0: 7948 MB, 7948206080 bytes
4 heads, 16 sectors/track, 242560 cylinders, total 15523840 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000dbfc6

        Device Boot      Start         End      Blocks   Id  System
/dev/mmcblk0p1            8192      122879       57344    c  W95 FAT32 (LBA)
/dev/mmcblk0p2          122880    15523839     7700480   83  Linux

The above shows the tail end of the output on my system. I can recognise the card by the size, 8 Gb (or 7948 as listed above) which is the only drive I have of this size, and its two partitions bearing in mind the fact that one of them is small and W95 FAT (LBA) and the other plain old Linux.

In the above, the suffixes ‘p1′ and ‘p2′ indicate the partitions on the card. The device itself is /dev/mmcblk0 and that’s what we need, the device name, not the partition names.

To take a backup, switch to your backup directory, where you intend to keep the copy of your SD card, and run the following command as the root user (or use sudo):

dd if=/dev/mmcblk0 of=Rpi_8gb_backup.img bs=2M 
3790+0 records in
3790+0 records out
7948206080 bytes (7.9 GB) copied, 1369.42 s, 5.8 MB/s

That’s it. The whole 8Gb SD card has been (slowly) copied to my computer. How do I know it worked?

Check the Backup File

The output above shows that there were no errors in copying the SD card to disk, but it’s always best to be sure that it will work when the time comes to carry out a restore.

The first step, as root – as ever – is to determine where the partitions begin. Because we have an image copy of the SD card, we can look at the file itself and see the partition table within.

The following commands will be carried out as root, or using the sudo command.

We first list the SD image file to see where the partitions begin and the sizes of the sectors. Fdisk reports sizes in sectors, but handily displays the sector size at the top.

fdisk Rpi_8gb_backup.img 
Welcome to fdisk (util-linux 2.21.2).
...
Command (m for help): p

Disk /home/norman/Backups/Rpi_8gb_backup.img: 7948 MB, 7948206080 bytes
255 heads, 63 sectors/track, 966 cylinders, total 15523840 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000dbfc6

                                  Device Boot      Start         End      Blocks   Id  System
/home/norman/Backups/Rpi_8gb_backup.img1            8192      122879       57344    c  W95 FAT32 (LBA)
/home/norman/Backups/Rpi_8gb_backup.img2          122880    15523839     7700480   83  Linux

We can see from the line starting with ‘Units’ or ‘Sector size’ that a sector is 512 bytes. We can also see that our two partitions start at sectors 8,192 and 122,880.

Multiplying these start sectors by 512 gives the start position in bytes. This works out at 4,194,304 and 62,914,560 bytes respectively.

On your Raspberry Pi, the first partition is mounted at /boot while the second is the root (/) mount point.

To check these, without needing access to the Pi, we simply proceed as follows on a Linux computer:

mkdir /mnt/root /mnt/boot
chmod a=rwx /mnt/root /mnt/boot

mount -t vfat -o loop,offset=4194304 /home/norman/Backups/Rpi_8gb_backup.img /mnt/boot

mount -t ext4 -o loop,offset=62914560 /home/norman/Backups/Rpi_8gb_backup.img /mnt/root

The above creates a couple of mount points (directories) and then mounts the first partition within the image file as a vfat file system on /mnt/boot and then mounts the second partition as an ext4 file system on /mnt/root.

The first two commands to create the mount points and set the permissions on them are only required once, the first time you carry out this exercise.

You can now see the files by opening a file manager and looking at the /mnt/boot and /mnt/root directories – you should see your various files as if you were looking on your Raspberry Pi.

Restore a Single File

This method of mounting a disc image as a device is useful for times when you manage to delete a file, accidentally of course, on your Pi, but you know that you have a backup. If the Pi is on your network, you can simply mount the backup image as above, locate the file and check that it is the one you want, then use scp or sftp to copy the file to your Pi, as demonstrated below:

mount -t ext4 -o loop,offset=62914560 /home/norman/Backups/Rpi_8gb_backup.img /mnt/root

scp /mnt/root/etc/network/interfaces pi@raspberrypi:
pi@raspberrypi's password: 
interfaces                                                                    100%  183     0.2KB/s   00:00

The file is now located in the home directory of the pi user. All you need to do to move it to the correct place is:

sudo mv interfaces /etc/network/
sudo chmod u=rw,go=r /etc/network/interfaces
sudo chown root:root /etc/network/interfaces

Obviously, if you are like me, you have already given your root user a password, so you could miss out the above and simply scp the file straight into the correct location. You would only need to set the permissions and not the ownership of the file afterward.

scp /mnt/root/etc/network/interfaces rooot@raspberrypi:
root@raspberrypi's password: 
interfaces                                                                    100%  183     0.2KB/s   00:00

sudo chmod u=rw,go=r /etc/network/interfaces

So there you have it, how to backup your SD card, how to mount it on the backup computer to check that it is ok, and as a bonus, how to extract a file (or files) for an individual recover. What else could we do?

Restore a Full Backup

This is as simple as initialising the SD card for the first time. You can restore a backup file to your SD card provided that the SD card is bigger or the same size as the image file.

As ever, the following commands are executed as root or by using the sudo command.

dd if=Rpi_8gb_backup.img of=/dev/mmcblk0 bs=2M

It takes a while, but it will restore in the end. All you have to do is wait for the copy to finish and then boot the Pi with the restored SD card in the slot.

Copy Your Current System to a Larger Card

We can also use the smaller backup files to initialize a larger SD card, if we were perhaps upgrading. This is what I had to do when I restored a 4 Gb backup to my 8 Gb card. Once that was done, I booted the Pi with the restored SD card in place, and when I logged in, executing sudo raspi-config and selecting the option to Expand root partition to fill SD card extended the current almost 4 Gb partition to the full almost 8 Gb size required (on the next reboot that is.)

You can see that it worked by executing the df command, which does not need to be executed as root!

df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          7.3G  1.6G  5.4G  23% /
/dev/root       7.3G  1.6G  5.4G  23% /
devtmpfs        109M     0  109M   0% /dev
tmpfs            22M  208K   22M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            44M     0   44M   0% /run/shm
/dev/mmcblk0p1   56M   25M   32M  44% /boot

The first and last entries in the above show that the root file system, mounted on / is 7.3 Gb while the /boot file system is 56 Mb in size. So, my 4 Gb restore is now happily running on an 8 Gb card and I have all the space available to use.

Gertboard Parts List

Norman — Thu, 18 Oct 2012 08:10:31 +0000

The Gertboard doesn’t come supplied with a parts list that you can cross reference. I created one for myself, and discovered a missing component which Farnell quickly replaced the very next day – thanks.

If you wish, you can download the Gertboard parts list here

Raspberry Pi Sound

Norman — Fri, 12 Oct 2012 15:10:59 +0000

Want to know how to redirect the sound from your Pi to either the HDMI or to the headphone socket? Read on …

Update – 11 February 2013

I’ve written a small command line utility – PiSound – to control the settings of the audio device on your RaspberryPi. You can download it from https://github.com/NormanDunbar/PiSound. Enjoy.

Deciding on the Output Device

The following command is all you need:

sudo amixer cset numid=3 n

Where the final ‘n’ is as follows:

0 = Auto – if HDMI is connected, use that, otherwise try the headphone socket.
1 = Sound goes to the headphone socket.
2 = Sound goes to HDMI socket.

If your current user is a member of the audio group, the sudo parts of the amixer commands is not required.

You can make sure that it works by running a command such as:

aplay /usr/share/sounds/alsa/Front_Left.wav

You should be able to hear the sound if you have set the correct output as above.

Have fun.

Raspbain 16th December 2012 – Update

It appears that something (a technical term) has gone wrong in the 16/12/2012 Raspbian release and/or after sudo apt-get update && sudo apt-get upgrade – which stops sound working.

You can tell if you are affected as follows:

$ sudo amixer controls

numid=4,iface=MIXER,name='Master Playback Switch'
numid=3,iface=MIXER,name='Master Playback Volume'
numid=2,iface=MIXER,name='Capture Switch'
numid=1,iface=MIXER,name='Capture Volume'

If you see the above, then you are affected and nothing you can do will allow you to redirect sound to the headphone socket. When I say nothing, I mean, nothing, except the following, as explained here, however, read the next section before you start removing stuff that you might need!

sudo apt-get purge --yes pulseaudio
...
sudo reboot

When your Pi comes back up, login and try again:

$ sudo amixer controls

numid=3,iface=MIXER,name='PCM Playback Route'
numid=2,iface=MIXER,name='PCM Playback Switch'
numid=1,iface=MIXER,name='PCM Playback Volume'

Now you can use the sudo amixer cset numid=3 1 command as described above, to direct the audio output to your headphones.

But I need PulseAudio …

You might be in a situation where you need to keep PulseAudio installed. What to do? The answer is simple, in all the calls to amixer add in a card selector such as -c 0.

Normally, the PCM card is card 0 (zero) and PulseAudio is card 1 (one). Somehow, PulseAudio sets itself as the default card. I haven’t bothered to discover how or where it does this yet, I deinstalled PulseAudio on my system.

$ sudo amixer -c 0 controls

numid=3,iface=MIXER,name='PCM Playback Route'
numid=2,iface=MIXER,name='PCM Playback Switch'
numid=1,iface=MIXER,name='PCM Playback Volume'

Hooray! If the above works for you, where leaving out the card select options -c 0 does not, then you must add -c 0 to all the amixer commands below.

Muting Sound

Numid=2 determines if sound is muted or not. To mute sound, regardless of its output device, do this:

$ sudo amixer cset numid=2 0

and to unmute the sound:

$ sudo amixer cset numid=2 1

Volume Control

Numid=1 allows you to set the volume. The range is slightly strange in that it runs from -10239 to +400 with +400 being the maximum. On my system, I have a pair of X-mini powered and amplified speakers attached. A minimum value of -1000 gives a quiet sound, 0 (zero) gives reasonable sound and 400 is a bit too loud.

You adjust the volume as follows:

$ sudo amixer cset numid=1  -- -1000

Please note the double hyphen. This is required in front of any parameter that has a leading hyphen. In this case, the volume setting I require is -1000, so the double hyphen says “the following is a value, even though it has a hypen, it is not another flag or option!”

You can use the double hyphen in front of positive numbers as well, without any adverse effects.

$ sudo amixer cset numid=1  -- 234

Positive values between 0 and 400 appear unchanged while negative values between -1 and -10239 are rounded up to 0 to -10238.

The only way to get -10239 is to mute the sound.

Of course, being human, it would be nice to set the volume to something easily figured out, like a percentage, wouldn’t it? This would be nice, for example:

$ sudo amixer cset numid=1 60%

No need to work out numbers in a weird range, no need for the double hyphens etc. Try it, it works! The range is obviously from 0% to 100%, anything outside of those boundaries will be limited to the appropriate percentage. Setting the volume to 0% effectively mutes the audio output.

What are my Settings?

You may, if you wish, view all the settings on your Pi with the following single command:

$ sudo amixer contents

numid=3,iface=MIXER,name='PCM Playback Route'
  ; type=INTEGER,access=rw------,values=1,min=0,max=2,step=1
  : values=1
numid=2,iface=MIXER,name='PCM Playback Switch'
  ; type=BOOLEAN,access=rw------,values=1
  : values=on
numid=1,iface=MIXER,name='PCM Playback Volume'
  ; type=INTEGER,access=rw---R--,values=1,min=-10239,max=400,step=0
  : values=-1000
  | dBscale-min=-102.39dB,step=0.01dB,mute=1

If you wish to find the settings for one control only, use the same numid as you used to cset the control, but read the setting with the cget command instead:

$ sudo amixer cget numid=3

numid=3,iface=MIXER,name='PCM Playback Route'
  ; type=INTEGER,access=rw------,values=1,min=0,max=2,step=1
  : values=1
scale-min=-102.39dB,step=0.01dB,mute=1

There doesn’t appear to be a way of fetching the current setting into a variable for use in, say, a bash script. Not unless you parse the data out of the returned string. The following python code will do this for you:

import os
...
volume = None
stdout = os.popen('amixer cget numid=1')
try:
    volume = stdout.read()
finally:
    stdout.close()

# At this point, volume (a string) and contains all the output from
# the amixer cget command. Extract the volume value, if no exceptions
# occurred. It is None if there was an exception

if volume:
    volume = volume.split(':')[1].split('\n')[0].split('=')[1]

# At this point, volume contains the volume setting as a string.
...

There isn’t, as far as I can find, any way of getting the current volume as a percentage. If that’s what you want or need, I’m afraid you will have to work it out – as a slight clue, the following python code might help:

...
# volume is a string holding the volume setting or is None. 
# We want it as an integer percentage, or -1 for errors..

if volume:
    percentage = int(((float(volume) + 10240) / 10640) * 100)
else:
    percentage = -1
...

You must float the string value or some calculations end up as zero percent due to the division by 10640.

Oracle Index Clustering Factor Explained

Norman — Fri, 12 Oct 2012 07:18:41 +0000

Some people are confused by the clustering factor of an index in an Oracle database. Be confused no more, just read this article by Tom Kyte on the subject.

Give Your Raspberry Pi Turbo Mode

Norman — Wed, 19 Sep 2012 16:29:50 +0000

And while you are at it, no warranty problems when you do so!

An announcement on the Raspberry Pi Foundation web site, read it here, explains that it is now possible to use Turbo mode on your Raspberry Pi without invalidating your warranty by over volting the device.

Please go and read it and then pop back here for instructions on adding Turbo mode to your own Raspberry Pi.

Back again? Ready to begin? Let’s do it!

The following assumes that you are running Raspbian as your chosen distro, and that you are logged in as the pi user.

sudo apt-get update
sudo apt-get upgrade
sudo raspi-config

    select the "update" option which updates raspi-config and exits.

sudo raspi-config
    select the new "configure overclocking" option and choose one.
    select finish.
    select yes to reboot now.

On reboot, your Raspberry Pi will be running [much] faster under load and, if it gets too hot, will slip back to normal non-turbo mode automagically.

It is possible that your raspberry Pi will not boot if the settings are too high for your system. If this is the case, hold down the shift key during the boot.

I set mine to full TURBO mode and it booted, but the CPUFreq Monitor (see below) shows 700 MHz rather than 1 GHz, however, this is because the Pi is now running in automatic gearbox mode – when the CPU usage gets above 85%, then it switches to a higher gear.

You can see this if you have the CPUFreq Monitor installed (see below) and, in a shell session run something like sudo apt-get update and while that is running, hover over the CPUFreq Monitor icon to see the speed changing up and down as the load comes and goes.

You can see the CPU Frequency in use in a console as per the following example:

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq
1000000

The output from which will be the frequency running now, in KHz. Divide by 1,000 to get the MHz frequency. The example above was taken while an sudo apt-get update was in progress. After the hard work was done, it dropped again:

cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq
700000

As mentioned above, if your Pi refuses to boot up with your chosen settings, simply reboot while holding down the shift key to skip the turbo mode setting, then sudo raspi-config to set it to a lower setting again.

You can, if you are running a GUI using XFCE, add the temperature sensor output and the current CPU frequency settings to your task bar, as follows:

Right-click the task bar and select “Panel settings”
Click on the “Panel Applets” tab
Click the “Add” button.
Select “Temperature Monitor” from the list, and click ADD.
Use the “Up” and “Down” buttons to put it where you want it to be.
Click the “Add” button.
Select “CPUFreq Frontend” from the list and click ADD.
Use the “Up” and “Down” buttons to put it where you want it to be.
Click the “Close” button.

You should see a couple of new items in the task bar. One is a two digit number (or hopefully two digits) in green text, and the other looks like a, well, I’m not sure what it looks like in my VNC session!

Anyway, hover the mouse over each and you will get told what’s what. The CPUFreq Monitor tells you what frequency you are running at and the temperature gauge stays green as long as the Raspberry Pi is running within temperature bounds. If the text turns red, it’s getting too hot and your Pi should shut down to a slower overclock/overvolt to reduce the temperature.

You can configure these applets by right-clicking on them, and selecting the applet’s own settings menu option (at the top of the menu).

Again, there is a console tool to allow you to see the current temperature:

cat /sys/class/thermal/thermal_zone0/temp
42236

The temperature reported in in thousanths of a degree Celsius. Divide by 1,000 to get degrees C. (Subtract 273 for degrees Kelvin!)

See the original article, links above, for full details of what else is included in this release. You can also read this article on setting up the other options in the new release.

You Are Not Allowed Science Any More

Norman — Mon, 17 Sep 2012 11:17:38 +0000

Brilliant! Read Tony Ballantyne’s excellent article. And enjoy.

The Dangers of Cheap USB Hubs

Norman — Sat, 25 Aug 2012 14:26:56 +0000

As a Raspberry Pi user, I figured I needed a USB hub to allow me to connect stuff (that’s a technical term by the way) to my Pi without overloading the two on board USB ports as these are limited to a maximum of 100 mA each. I purchased a cheap 7 port externally powered hub and it arrived by next day delivery.

The USB 2.0 specification says that anything advertising compliance should, amongst other things be able to supply 500 mA minimum on each and every port while externally powered and should not feed any power back to the computer via the USB interconnect cable – that’s the one between the computer and the hub.

This is a link to a USB hub that is identical to the one I purchased and there are plenty of others that look exactly like this on Ebay as well as other places. Please note, I did not purchase my hub from Amazon and I certainly didn’t pay nearly £20 for it either! It looks a whole lot better in the Amazon photo that it does in real life, I can assure you.

They are available new, on Ebay, from around £3.00 to about £9.00. Mine cost £7.75 and was badged with the name Dynamode (only on the packaging though!) and claimed to be fully USB 2.0 compliant. Ha! They all look remarkably like the image to be found on this page somewhere around here! The text on top is “[HUB]” if you can’t make it out.

Don’t buy one of these! (Click for a bigger image.)

Ok, if you read my review on Amazon at the above link, you’ll know what was wrong with it. If you can’t be bothered, read on…

When plugged into the Pi with the USB interconnect cable, and the hub was then powered on (but not yet the Pi), the power LED on my Pi lit up. That should never happen and shows that the hub is allowing power to be sourced from the hub rather than only sunk from the Pi. That fails section 7.2.1 of the USB 2.0 standard and means that the hub is not USB 2.0 complaint at all.

In addition, removing the interconnect cable and powering the Pi from one of the USB ports, didn’t supply enough power to allow the Pi to boot. Yet again, this fails the USB 2.0 standard which clearly says that all ports must be able to provide a minimum of 500 mA when the hub is externally powered (not on batteries though). It should perhaps be noted that the Pi requires 500 mA to boot and an additional 100 mA per “in use” USB port. My Pi had nothing plugged in to the USB so only requires 500 mA.

That’s two failures so far!

Even worse, given that this was a 7 port hub, each of those ports should have been able to supply a minimum of 500 mA and yet, the power supply that came with the hub was rated at 1000 mA – which is only just enough for a two port hub and nowhere near enough for a 7 port hub.

And it gets worse!

With this hub attached to the Pi using the interconnect cable, the Pi refused to boot. Removing the hub and booting the Pi was fine, but on connecting the hub, all networking stopped functioning! My SSH and VNC sessions dropped out and attempting to reconnect just failed. Removing the hub allowed networking to work again.

So, all in all, it’s not worth the money and it doesn’t do what it says on the packaging. These things are best avoided. I took it back to the shop for a refund and explained why I was returning it. All the details were marked down on their system and it’s just possible that they will not be selling these any more.

These things do work with laptops and such like, but are you willing to connect one to your laptop when it supplies power back down the interconnect cable? If your laptop (or computer) doesn’t have a diode present to prevent a back feed, who knows what it might damage? The Raspberry Pi doesn’t have such a diode, but why should it have one? The standard is completely clear on the matter – no power down the interconnect!

That Power Supply Again

It appears that someone with better electronic skills than myself has taken the power supply on one of these hubs apart. Watch the video and listen carefully to the commentary on this You Tube video. (Opens in a new window/tab.) Stick with it to the end and be amazed at how close you come to stuffing mains voltage down your USB devices!

A Working, Tested USB Hub

Buy this hub instead!

I have finally found a working hub for my Raspberry Pi. I purchased it from ModMyPi and this link takes you directly to the product in their shop. Delivery was next day by Royal Mail and it just works.

My Pi is running now with a 60Gb USB hard drive, a USB Floppy Drive (don’t ask!) and is powered by the hub itself. The power supply is 2A and the hub is a 4 port one which works out at 500 mA per port which is exactly correct according to the standards. There is no back feed of power via the interconnect – so, it’s perfect.

Using Oracle 11g Adrci for Incident Reporting

Norman — Wed, 22 Aug 2012 11:49:56 +0000

Adrci is a new tool in Oracle 11g which makes life a little easier when gathering evidence to send off to Oracle Support, but it can make life easier when you simply wish to view the alert log, for example.

As ever, you need to be logged in to the database server and have the environment set in the normal oraenv manner. This is how we used to do it in the old, pre 11g, days:

SQL> connect / as sysdba
connected

SQL> show parameter background_dump_dest

NAME                  TYPE    VALUE
--------------------  ------  -----------------------------------------
background_dump_dest  string  /srv/xxx/oradata/yyyyy/diag/rdb
                              ms/yyyyy/yyyyy/trace
SQL> exit

$ view /srv/xxx/oradata/yyyyy/diag/rdbms/yyyyy/yyyyy/trace/alert_yyyyy.log

In modern times, with 11g of course, we no longer have to do this, as we have adrci instead, as follows:

$ adrci

ADRCI: Release 11.2.0.3.0 - Production .....
...
ADR base = "/srv/xxx/oradata/yyyyy"

adrci> show alert

That’s all there is to it. The alert log is, on my server, displayed in the vi editor and when you quit with the :q command, you return to adrci.

While you are there, you can check for any incidents recorded in the alert log:

adrci> show incident

ADR Home = /srv/xxx/oradata/yyyyy/diag/rdbms/yyyyy/yyyyy
**********************************************************************
INCIDENT_ID  PROBLEM_KEY       CREATE_TIME
------------ ----------------- ---------------------------------
145          ORA 227           2012-08-06 16:22:40.026000 +01:00
40165        ORA 600           2012-08-10 13:14:23.033000 +01:00

Given that ORA-600, I can get more detail about it with the following command:

show incident -mode detail -p "incident_id=40165"

ADR Home = /srv/xxx/oradata/yyyyy/diag/rdbms/yyyyy/yyyyy
**********************************************************************

**************************************
INCIDENT INFO RECORD 1
**************************************
  INCIDENT_ID       40165
  STATUS            ready
  PROBLEM_ID        4
...

The -mode option can be one of:

Basic – simply lists the appropriate line(s) that you would see in a show incident command output. Not much use to be honest.
Brief – gives a bit more detail that basic. Doesn’t include details of any trace files created for this incident.
Detail – gives lots of information including details of all trace files created for this incident.

The -p option is simply a where clause on any of the columns in the incident table. You can obtain a list of columns as follows:

adrci> describe incident

Name          Type          NULL?
------------- ------------- ------
INCIDENT_ID   number
PROBLEM_ID    number
CREATE_TIME   timestamp
CLOSE_TIME    timestamp
STATUS        number
FLAGS         number
....

The -p options can be ANDed and/or ORed together:

adrci> show incident -p "incident_id = 40165 or incident_id = 145"

Given that incident 40165 is an ORA-600, it should be sent off to Oracle Support for diagnosis, so I need to gather all the evidence together. Remember how that was done in the old days? Well, this is how simple it is with adrci:

First, create a new package:

adrci> ips create package
Create package 7 without any contents, correlation level typical

That creates a blank package, with nothing in it. You now add all the incidents you need to add:

adrci> ips add incident 40165 package 7
Added incident 40165 to package 7

That has added the incident to the package’s metadata only, there are still no actual package files created. You may add further incidents to the package as desired.

When done adding incidents, generate a physical package for support:

adrci> ips generate package 7
Generate package 7 in /home/oracle/IPSPKG_20120820104439_COM_7.zip, mode complete

The zip file name created contains everything Oracle Support will need, all the trace files, the alert log, various metadata files and so on. This file can be uploaded to Oracle Support.

The file is created in the current working directory. If you want to specify where to put it, the following command will help:

adrci> ips generate package 7 in '/srv/xxx/oradata/yyyyy'
Generate package 7 in /srv/xxx/oradata/yyyyy/IPSPKG_20120820104842_COM_7.zip, mode complete

That concludes this very brief overview of the adrci utility. For more details check out the Database Administrator’s Guide, Chapter 9 – Managing Diagnostic Data Have fun.

Installing Oracle Multimedia on 11g

Norman — Fri, 17 Aug 2012 12:42:41 +0000

Installing Oracle Multimedia, which is required for Spatial and/or Locator is quite simple.

All of the following must be carried out while logged in as a SYSDBA user.

SQL> spool ordinst.log
SQL> @?/ord/admin/ordinst SYSAUX SYSAUX
...
...
SQL> spool off

SQL> spool catim.log
SQL> @?/ord/im/admin/catim
...
...
SQL> spool off

Lots of stuff will scroll up the screen but will also be copied to the spool files named. Check those for obvious errors, then check to see if it worked as follows:

SQL> select comp_id, version, status
  2  from dba_registry
  3  where comp_id = 'ORDIM';

COMP_ID    VERSION        STATUS
---------- ----------     ---------
ORDIM      11.2.0.3.0     VALID

You can also force check the validity as follows, should you ever need to:

SQL> set serveroutput on

SQL> execute sys.validate_ordim;
PL/SQL procedure successfully completed.

If there are any invalid objects in the ORDIM component, the above procedure will display a message about them and set the status column in the DBA_REGISTRY to INVALID. You should check the status after running the above.

Rename an Oracle 10g or 11g Database – Part 2

Norman — Thu, 09 Aug 2012 15:21:01 +0000

So, you renamed your database using the nid utility as outlined here but now you need (or want) to change all the file system names to suit. Read on.

In the following example, we have two mount points for the database. Files on these mounts are spread all over a pile of separate discs making up the LUN – so it’s not as bad as it looks!

The two mounts are wrongly named at the moment since we changed the database name using nid and we would like to tidy things up. The current names are:

/srv/old_name/oradata/old_sid/
/srv/old_name/flashback_area/old_sid

What we would like to see after the renaming is the following:

/srv/new_name/oradata/new_sid/
/srv/new_name/flashback_area/new_sid

First of all, you need a current pfile. If you don;t have one already (from the rename exercise) then create one as follows:

create pfile='/home/oracle/initnew_sid.ora' from spfile;

We also need a control file trace taking, so we do this next:

alter database backup controlfile to trace as '/home/oracle/new_sid_controlfile.sql';

It should be obvious that you wouldn’t want the pfile and control file trace to be located on the file system that is about to be moved. If they are, you will have to wait for the Sys Admins to complete the remounting exercise before you can do your own editing in preparation for restarting the database.

The next thing to do is backup the database. Shut it down and take a cold backup in your preferred manner – RMAN or some other utility. Better safe than sorry after all.

In a shell session, as the oracle user, remove (or move) the initnew_sid and spfilenew_sid.ora files, if present, from $ORACLE_HOME/dbs:

rm $ORACLE_HOME/dbs/initnew_sid.ora
rm $ORACLE_HOME/dbs/spfileold_sid.ora

Edit the newly generated parameter file as required and move it from /home/oracle/initnew_sid.ora to $ORACLE_HOME/dbs/initnew_sid.ora. Things to change will include some or all of the following:

audit_file_dest
control_files
db_recovery_file_dest
diagnostic_dest
log_archive_dest
etc

Anything that has been moved (or will be moved) by renaming the mount points will require changing.

Next, the control file trace that was created needs to be edited. There is much waffle to be removed and a choice of script to generate. Edit the file /home/oracle/new_sid_controlfile.sql and:

Find the section of the trace file that is applicable to your wish to start the database with RESETLOGS or not. For NORESETLOGS it is the first section, near the top of the trace, otherwise it is the second section near the bottom.
For my own databases, I have chosen not to reset the logs, so I use the top section of the trace file.
Delete all the comment lines above the section you want. You need to start the script from the STARTUP NOMOUNT command in the appropriate section.
Scroll down, but retain all the lines from (and including) STARTUP NOMOUNT until you find a comment line that looks something like -- End of tempfile additions. Delete that line, and everything else to the end of file.

You should now be in possession of a script that begins with STARTUP NOMOUNT and ends with a few lines adding your TEMPFILEs back into your temporary tablespace9s).

Run a global search and replace to ensure that all filenames match with the new locations.
If the database is in ARCHIVELOG mode, pay particular attention to the commented out commands to ALTER DATABASE REGISTER LOGFILE and make sure you enter the full path to an existing log file. You can ignore this part if your database is running in NOARCHIVELOG mode.
Save the file.

When the Sys Admins have given you back the file systems mounted on their new locations, it’s a simple matter to:

Connect to the database as sysdba.
@/home/oracle/new_sid_controlfile.sql

When that has finished, everything should now be in order. You may create a new spfile in the usual manner:

create spfile='/home/oracle/spfilenew_sid.ora' from pfile; shutdown host cp /home/oracle/spfilenew_sid.ora $ORACLE_HOME/dbs/ startup

Nothing to it!

Beware of Listener Port 1521 and Dynamic Registration

Norman — Fri, 03 Aug 2012 10:04:46 +0000

As you already know, an Oracle database’s PMON process will register your database with a listener without you having to do anything about it. However …

This will only happen if the listener in question is running on port 1521. And it doesn’t have to be named LISTENER either – as I mistakenly thought, it only has to be port 1521.

If you have a listener running on this port, and you have configured other listener(s) to listen on different ports, you will still have them dynamically register with the listener running on port 1521.

Lsnrctl status shows this for a non-1521 listener on the same server as a listener running on port 1521 (names changed to protect the guilty):

...
Listening Endpoints Summary...

(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=test_server)(PORT=1525)))
Services Summary...
Service "test5" has 1 instance(s).
  Instance "test5", status UNKNOWN, has 1 handler(s) for this service...
The command completed successfully

For the listener running on port 1521, we can see it’s grabbed everything!

...
Listening Endpoints Summary...

(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=test_server)(PORT=1521)))
Services Summary...
Service "test1" has 1 instance(s).
  Instance "test1", status READY, has 1 handler(s) for this service...
Service "test1XDB" has 1 instance(s).
  Instance "test1", status READY, has 1 handler(s) for this service...
Service "test2" has 1 instance(s).
  Instance "test2", status UNKNOWN, has 1 handler(s) for this service...
Service "test3.world" has 1 instance(s).
  Instance "test3", status READY, has 1 handler(s) for this service...
Service "test4" has 1 instance(s).
  Instance "test4", status READY, has 1 handler(s) for this service...
Service "test5" has 1 instance(s).
  Instance "test5", status READY, has 1 handler(s) for this service...
The command completed successfully

If you want to prevent this from happening, you can add the following to the listener.ora for the listener you wish to have listening on port 1521:

DYNAMIC_REGISTRATION_lsnr_name = off

That way, the listener is prevented from accepting dynamic registration requests from the other databases’ PMON process.

How to Screw Up DB Upgrade Assistant

Norman — Thu, 21 Jun 2012 15:58:47 +0000

It was my own fault, but in case it proves even slightly useful….

I was upgrading from 11202 to 11203 Enterprise using the DB Upgrade Assistant utility. When I said ‘go do it’ it went off, chugged for a bit, then barfed. DBUA informed me that the database wasn’t running. I checked, it was.

Cutting a long story short, I checked the indicated logfile and discovered that DBUA had connected to the database but then got a couple of errors telling it that ‘oracle was not available’. Hmm.

Turns out that I’d added a couple of SQL statements to glogin.sql, with

alter session set nls_date_format = 'dd/mm/yyyy hh24:mi:ss';

being one of the offending statements. This works perfectly as long as you connect while the database(s) are open but, if a database is shut, and you login as sysdba to start it, glogin.sql is still executed, and because the database is down, the SQL statement(s) fail.

DBUA picked up the failure and refused to carry on.

Removing the SQL statement(s) from glogin.sql fixed the problem.

The moral to this tale is simple, don’t put SQL in glogin.sql (or login.sql) if you ever shut down your databases!

In case you are wondering, glogin.sql lives in $ORACLE_HOME/sqlplus/admin and will be executed on each successful connection to a database with SQL*Plus. Login.sql will also be executed on every successful connection to a database (at least, from 10g onwards) after glogin.sql, but only if it is found on $SQLPATH.

Oracle RMAN for Beginners – Part 8

Norman — Tue, 15 May 2012 15:24:33 +0000

In the previous instalment of this exciting series, we completed the full (complete) recovery of the database, tablespaces and data files, as well as looking at recovering individual blocks.

In this article, we will perform incomplete recovery where we restore and recover the databases to a specific point in time that is previous to “now”.

Note that in the following examples I’m using the until time option and specifying a date and time. You can restore to a particular SCN, or archive log number etc. See the manual for all the options.

It is possible that data will be lost with this kind of recovery. It is not possible to perform incomplete recovery on tablespaces or data files, you are limited to restoring and recovering the entire database. Although, this is not quite true, read on …

Incomplete Recovery of the Database

As the whole database is being restored, the SYSTEM and UNDO tablespaces will be restored and recovered, so the database will need to be in a mounted state. You cannot carry out incomplete recovery on an open database.

First of all, let’s add a new row to our test table which we use to show progress etc.

SQL> alter session set nls_date_format='dd/mm/yyyy hh24:mi:ss';
Session altered.

SQL> insert into test values (sysdate);
1 row created.

SQL> commit;
Commit complete.

SQL> select * from test order by 1;

A
-------------------
06/02/2012 20:25:11
16/02/2012 17:26:20
17/02/2012 15:03:42
17/02/2012 15:54:49
07/03/2012 11:21:53
19/03/2012 20:58:18
15/05/2012 13:44:29

7 rows selected.

Now, restore and recover the database back to a time that is about 15 minutes ago. The first stage is to get the database into a mount state.

rman target /

RMAN> shutdown;

database closed
database dismounted
Oracle instance shut down

RMAN> startup mount

connected to target database (not started)
Oracle instance started
database mounted

Total System Global Area     768331776 bytes

Fixed Size                     2230360 bytes
Variable Size                213911464 bytes
Database Buffers             549453824 bytes
Redo Buffers                   2736128 bytes

Next, we restore the database to our chosen time.

RMAN> restore database until time "to_date('15/05/2012 13:30:00','dd/mm/yyyy hh24:mi:ss')";

Starting restore at 15/05/2012 14:08:47
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=19 device type=DISK

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00002 to /srv/nffs/oradata/ant12/data/sysaux01.dbf
...
...
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
Finished restore at 15/05/2012 14:13:04

I’ve trimmed the usual RMAN verbose output from the above to save you falling asleep reading it!

Next, we need to recover the database to the same time.

RMAN> recover database until time  "to_date('15/05/2012 13:30:00','dd/mm/yyyy hh24:mi:ss')";

Starting recover at 15/05/2012 14:16:35
using channel ORA_DISK_1

starting media recovery

archived log for thread 1 with sequence 12 is already on disk as file /srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_12_7mwyxlk8_.arc
...
...
media recovery complete, elapsed time: 00:00:36
Finished recover at 15/05/2012 14:17:24

Again, I’ve trimmed the output. RMAN will restore to disc any archived logs it requires to carry out the recovery if it determines that they are not available online.

The final stage is to reopen the database. You must use the resetlogs option because you’ve carried out an incomplete recovery.

RMAN> alter database open resetlogs;
database opened

So, the restore and recovery went well, did it really work?

SQL> conn norman/norman
Connected.

SQL> alter session set nls_date_format='dd/mm/yyyy hh24:mi:ss';
Session altered.

SQL>  select * from test;

A
-------------------
07/03/2012 11:21:53
19/03/2012 20:58:18
06/02/2012 20:25:11
16/02/2012 17:26:20
17/02/2012 15:03:42
17/02/2012 15:54:49

6 rows selected.

So, our most recent row in the table is no longer present, we now have 6 rows instead of the 7 that we had at the start of this recovery. The database has been restored back in time.

You will note that in both the restore database and the recover database commands, I had to specify the same time. There is a short cut that you can use to save typing.

RMAN> run {
2> set until time = "to_date('15/05/2012 13:30:00','dd/mm/yyyy hh24:mi:ss')";
3> restore database;
4> recover database;
5> }

executing command: SET until clause

Starting restore at 15/05/2012 14:02:23
using channel ORA_DISK_1
...
...
Finished restore at 15/05/2012 14:02:24

Starting recover at 15/05/2012 14:02:24
using channel ORA_DISK_1
...
...
media recovery complete, elapsed time: 00:00:02

Finished recover at 15/05/2012 14:02:26

Now you can open the database in resetlogs mode as above.

Restoring a Database Through a Resetlogs

In the past it wasn’t possible to restore a database that had been opened with resetlogs specified. RMAN used to treat this as a new incarnation of the database, so you had to make sure you took a fresh backup as soon as the database was opened.

RMAN in 11g is much more forgiving and it is now possible to restore through a resetlogs. You do this by restoring a backup from the previous incarnation and do a recover. If you watch the recovery phase carefully, you will see the archived logs files being applied from before and after the resetlogs.

The following assume that you haven’t taken a backup of the new incarnation yet. If you have, you must restore the controlfile from a named backup as opposed to letting RMAN work it out.

The first step is to restore the controlfile.

RMAN> shutdown immediate;

database closed
database dismounted
Oracle instance shut down

RMAN> startup nomount;

connected to target database (not started)
Oracle instance started

Total System Global Area     768331776 bytes

Fixed Size                     2230360 bytes
Variable Size                213911464 bytes
Database Buffers             549453824 bytes
Redo Buffers                   2736128 bytes

RMAN> restore controlfile from autobackup;

Starting restore at 15/05/2012 15:12:57
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=17 device type=DISK
...
...
Finished restore at 15/05/2012 15:13:03

RMAN> alter database mount;

database mounted
released channel: ORA_DISK_1

Now that we are on an old controlfile and the database is mounted, we can restore and recover the database in the normal manner.

RMAN> restore database;

Starting restore at 15/05/2012 15:15:06
Starting implicit crosscheck backup at 15/05/2012 15:15:06
allocated channel: ORA_DISK_1
...
...
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
Finished restore at 15/05/2012 15:19:27

Now all we have to do is the recover. As mentioned above, watch the thread and sequence numbers closely.

RMAN> recover database;

Starting recover at 15/05/2012 15:19:36
using channel ORA_DISK_1

starting media recovery
...
...
archived log ... thread=1 sequence=8
archived log ... thread=1 sequence=9
...
...
archived log ... thread=1 sequence=20
archived log ... thread=1 sequence=1
media recovery complete, elapsed time: 00:00:52
Finished recover at 15/05/2012 15:20:42

You can hopefully see, in the above, that the sequence number dropped from 20 to 1 as we recovered through the resetlogs. Talking of which, we need another one now.

RMAN> alter database open resetlogs;

database opened

Tablespace Incomplete Recovery

It was never possible to do a tablespace point in time recovery without cloning the database first and using the clone to export the data. However, with 11g it is now possible to perform incomplete recovery on a tablespace (or tablespaces) as RMAN takes care of all the cloning etc.

First of all, create a working area for the clone, as root.

# mkdir /media/oracle_backups/working_aux
# chown oracle:oinstall /media/oracle_backups/working_aux/
# ls -l /media/oracle_backups/
total 24
drwxr-xr-x 2 oracle oinstall  4096 Feb 17 16:51 ant12
drwx------ 2 root   root     16384 Feb  6 08:07 lost+found
drwxr-xr-x 2 oracle oinstall  4096 May 15 15:34 working_aux

Next, check our test table to see how far back we can recover the users tablespace.

SQL> conn norman/norman
Connected.

SQL> alter session set nls_date_format='dd/mm/yyyy hh24:mi:ss';
Session altered.

SQL> select * from test;

A
-------------------
07/03/2012 11:21:53
19/03/2012 20:58:18
06/02/2012 20:25:11
16/02/2012 17:26:20
17/02/2012 15:03:42
17/02/2012 15:54:49

6 rows selected.

Obviously your decision criteria for the actual time to restore back to will be a little more scientific than mine! But I’m going to select a date of just after 16/02/2012 17:26:20, lets call it 16/02/2012 17:30:00.

The first problem we may happen across is whether there are any objects in the tablespace to be restored which have constraints etc in other tablespaces. We check the SYS.TS_PITR_CHECK.

SQL> conn / as sysdba
Connected.

SQL> select * from ts_pitr_check
  2  where (ts1_name = 'USERS' and ts2_name <> 'USERS') or
  3 (ts2_name = 'USERS' and ts1_name <> 'USERS');
no rows selected

In this case as we are about to restore the users tablespace, we have to see if there are relationships from users to other tablespaces and also if there are relationships from other tablespaces to users.

There are none, but if there had been, we would either need to temporarily drop those constraints or include the other tablespace(s) in the recovery.

The next problem is those objects that exist in the database now, that didn’t at the time of the recovery point in time. Again we can find a list of those by checking SYS.TS_PITR_OBJECTS_TO_BE_DROPPED and giving the date and time we intend to use.

SQL> select tablespace_name, owner, name
  2  from ts_pitr_objects_to_be_dropped
  3  where tablespace_name = 'USERS'
  4  and creation_time > to_date('16/02/2012 17:30:00','dd/mm/yyyy hh24:mi:ss');
no rows selected

So far so good. Had there been any objects listed, we would need to run an export of those objects using data pump to preserve them across the recovery.

We are now ready to carry out a point in time restore of the users tablespace to the working area instead of to the normal database area. You do not need to take the tablespace offline, RMAN does that for you.

Note, in the following command recover is correct. Do not use restore as it will not work.

RMAN> recover  tablespace users until time "to_date('16/02/2012 17:30:00','dd/mm/yyyy hh24:mi:ss')"
2>  auxiliary destination '/media/oracle_backups/working_area';

Starting recover at 15/05/2012 15:55:52
using channel ORA_DISK_1

Creating automatic instance, with SID='qvbt'

initialization parameters used for automatic instance:
db_name=NORM
db_unique_name=qvbt_tspitr_NORM
...
...
Removing automatic instance
Automatic instance removed
Finished recover at 15/05/2012 15:58:42

You will note that there is a lot of output from RMAN as it builds a clone database and recovers the tablespace to the specified time.

You cannot recover a tablespace to a time previous to the last resetlogs time. If you try, you will get the following error message:
RMAN-03002: failure of recover command at 05/15/2012 15:55:53
RMAN-20207: UNTIL TIME or RECOVERY WINDOW is before RESETLOGS time

After the tablespace has been cloned and recovered, you must back it up and then bring it online.

RMAN> backup tablespace users;
...
RMAN> sql 'alter tablespace users online';
...

If you do not have an RMAN catalogue in use, you are not permitted to carry out more than one of these tablespace point in time recovery operations because the controlfile no longer knows about the previous incarnations of the recovered tablespace(s).

This is one of the reasons why you must backup the tablespace immediately after recovering it.

If, on the other hand, you do use a catalogue, then multiple recoveries of the same tablespace(s) are permitted.

Installing Locator or Spatial on 11g

Norman — Tue, 15 May 2012 09:53:26 +0000

Locator is “Spatial Lite” if you wish, and costs nothing. It can be installed in Standard or Enterprise Editions with no additional licensing costs. You cannot do everything in Locator that you can in Spatial – but what do you expect for free?

To install Locator you need to be aware that things changed at 11g, so what used to work on 10g no longer does on 11g and can lead to you having a huge mess of objects and types to hunt down and remove from your SYS account – ask me how I know!

Note: If you have already installed Oracle Multimedia, the MDSYS user will already exists. In this case, simply skip the create user command in the following.

SQL> connect / as sysdba

SQL> create user mdsys identified by secret
  2  default tablespace sysaux;

SQL> @?/md/admin/mdprivs.sql

SQL> connect mdsys/secret
SQL> @?/md/admin/catmdloc

SQL> connect / as sysdba
SQL> alter user mdsys
  2  account lock
  3  password expire;

You can read the details about Locator in the file $ORACLE_HOME/md/doc/README_LOCATOR.txt – but don’t expect too much assistance, it doesn’t even mention how to install it!

You won’t be able to find Locator in DBA_REGISTRY nor will you be able to find it in V$OPTION either.

SQL> col parameter format a20
SQL> col value format a5

SQL> select parameter, value
  2  from v$option
  3  where parameter like 'Spatial%'
  4  or parameter like 'Locator%';

PARAMETER            VALUE
-------------------- -----
Spatial              FALSE

You can see that Locator doesn’t turn up in the output – that’s because it’s not an option, unlike Spatial.

My Oracle Support (MOS) has a document that explains how to detect if Locator is installed. The document id is 357943.1. Unfortunately, the process explained doesn’t work. Sigh.

You cannot even assume Locator is installed if Spatial is not installed and you have a schema named MDSYS which owns objects, because installing Oracle Multimedia creates the MDSYS user if it is not there already.

Installing Spatial requires that you are running Enterprise Edition and that you didn’t deselect it when installing the software originally. If you did, you need to re-run the installer and when you see the Enterprise Options button, click it and select the Spatial component.

If you don’t do this then the file catmd.sql and it’s associated helpers, will not be copied to $ORACLE_HOME/md/admin.

Spatial costs extra dosh and must be licensed separately from your Enterprise Edition software.

SQL> connect / as sysdba

SQL> create user mdsys identified by secret
  2  default tablespace sysaux
  3  account lock
  4  password expire;

SQL> @?/md/admin/mdprivs.sql

SQL> @?/md/admin/catmd

You can read the details about Spatial in the file $ORACLE_HOME/md/doc/README.txt.

As with Locator, you won’t find Spatial in DBA_REGISTRY so, as before, you must look in V$OPTION.

SQL> col parameter format a20
SQL> col value format a5

SQL> select parameter, value
  2  from v$option
  3  where parameter like 'Spatial%';

PARAMETER            VALUE
-------------------- -----
Spatial              TRUE

Oracle Text aka CONTEXT – Installing on 11g

Norman — Mon, 14 May 2012 15:14:37 +0000

It’s supposed to be installed by default, according to the documentation, but for some reason or another, I managed to build a brand new 11.2 database, on Linux, with no CTXSYS user present.

Installing Context

Here’s how to install Oracle Text and the English language defaults, into an 11.2 database.

SQL> connect / as SYSDBA
SQL> spool ctxsys_installation.log

-- Parameters are password, default t/s, temp t/s, don't lock the account. 
SQL> @?/ctx/admin/catctx.sql secret SYSAUX TEMP NOLOCK

SQL> connect CTXSYS/secret
SQL> @?/ctx/admin/defaults/dr0defin.sql "ENGLISH";

SQL> connect / as SYSDBA
SQL> alter user ctxsys account lock password expire;

SQL> spool off

Removing Context

Here’s how to remove Oracle Text from an 11.2 database.

SQL> connect / as SYSDBA
SQL> spool ctxsys_removal.log

SQL> @?/ctx/admin/catnoctx.sql

SQL> spool off

I’m Quite Pleased With This Photo

Norman — Mon, 14 May 2012 13:11:46 +0000

We have foxes in our garden in the mornings and evenings. We seem to have a dog and vixen and three cubs. This morning they were cavorting around in the sun and I managed to get a few pictures taken before they headed off to wherever they lie up during the day.

This is my favourite shot:

Mother fox keeps an eye on me watching her and her cubs

Next up, we have one of the vixen and one cub:

One of the cubs comes accross to make sure "mum" is ok

Then one of the cubs playing together. They are so quick, it’s hard to get a decent focus. This shot is definitely out of focus, but high on cuteness!

Three fox cubs playing in our bushes

And finally, for now anyway, one of the whole family together. Something in the vixen’s look tells me she might be a tad fed up of all these kids!

The vixen looks perplexed at all those cubs running around

By the way, all shots taken on a Konica Minolta Dynax 5D fitted with a Minolta 75-300 Zoom and a 2* doubler. As the camera won’t quite autofocus at the top end of the zoom with the doubler attached, I was on manual focus.

These images are poor quality jpegs rather than the high quality raw format we normally shoot in.

No foxes were harmed in the taking of these photos.

Weird Error Dialogue when Installing Oracle 11.2.0.3 Client?

Norman — Tue, 24 Apr 2012 14:28:47 +0000

When installing the Pro*C compiler stuff (technical term) from the 11.2.0.3 client install disc (it’s in zipfile 4 or 7 in case you need to know!) I hit a strange error almost immediately the installer started the GUI.

I don’t have a screen dump, but the error message was the following:

vbOEL.dunbar-it.co.uk:vbOEL.dunbar-it.co.uk

In other words, my fully qualified Linux x86-64bit server name, twice. Interesting. No buttons were present to allow me to carry on, abort etc. Just an OK button which is not much help as it bales out of the installer when clicked.

Googling and DuckDuckGoing (will that ever be a verb?) found me nothing.

As it turned out, I was using the wrong CD as I should have been putting in the 11.2.0.2 version. Once I mounted the correct CD image, I started again. This time I got an error dialoge with a bit more information:

[INS-06101] IP address of localhost could not be determined.
Are you sure you wish to continue?

And this time, I have buttons for YES and NO. I clicked YES and all was well.

It looks like 11.2.0.3 is a tad flaky in the error message/dialogue department.

Cheers,
Norm.

Oracle 9i, HP-UX and ORA-12505 Drives Me Mad!

Norman — Mon, 23 Apr 2012 16:06:02 +0000

I’ve just created a new database on an HP-UX server. The database is Oracle 9i (yes, I know, I know!) and no matter what I do, I can’t connect via the listener without getting the dreaded ORA-12505: TNS:listener could not resolve SID given in connect descriptor error message.

I’ve done this lots of times in the past, but for some reason, I can’t get it to work today. It’s driving me mad!

The reason I have to use a local_listener parameter on HP-UX is because the hostname and the uname -n commands don’t return the same result as the hostname is longer than 8 characters:

SQL> !hostname
myserver0011

SQL> !uname -n
myserver

So, we have to use a local_listener setting in the spfile, and register the database with that listener because we can’t connect with user/password@alias otherwise.

SQL> connect my_user/secret@my9idb
ERROR:
ORA-12505: TNS:listener could not resolve SID given in connect descriptor
Warning: You are no longer connected to ORACLE.

Hmmm, it works fine without using the listener:

SQL> conn / as sysdba
Connected.

Some sanity checks:

SQL> show parameter db_name

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
db_name                              string      my9idb

SQL> show parameter local_listener

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
local_listener                       string      (ADDRESS=(PROTOCOL=TCP)(HOST=m
                                                 yserver0011)(PORT=1556))

SQL> !tnsping my9idb

TNS Ping Utility for HPUX: Version 9.2.0.6.0 - Production on 23-APR-2012 16:12:30

Copyright (c) 1997 Oracle Corporation.  All rights reserved.

Used parameter files:
/opt/oracle/product/9.2.0.6/db/network/admin/sqlnet.ora

Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=myserver0011)(PORT=1556)) (CONNECT_DATA=(SID=my9idb)(SERVER=DEDICATED)))
OK (10 msec)

SQL> !ping myserver0011
PING myserver0011: 64 byte packets
64 bytes from 10.55.127.122: icmp_seq=0. time=0. ms
64 bytes from 10.55.127.122: icmp_seq=1. time=0. ms
64 bytes from 10.55.127.122: icmp_seq=2. time=0. ms

----myserver0011 PING Statistics----
3 packets transmitted, 3 packets received, 0% packet loss 
round-trip (ms)  min/avg/max = 0/0/0

SQL> !lsnrctl services lsnr_my9idb

LSNRCTL for HPUX: Version 9.2.0.6.0 - Production on 23-APR-2012 16:15:51

Copyright (c) 1991, 2002, Oracle Corporation.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=myserver0011)(PORT=1556)))

The listener supports no services
The command completed successfully

The clue was in the second last line, the listener supports no services. But I missed it the first time I checked, and the second, third …..

It eventually dawned on me:

SQL> alter system set service_names='my9idb' scope=both;
System altered.

SQL> alter system register;
System altered.

SQL> conn myuser/secret@my9idb
Connected.

Result!

It appears that the alter system register command isn’t required, as soon as I set the service_names parameter, the listener seems to notice. However, I’m taking no chances after today!

All usernames, passwords, IP addresses and server names in this article have been disguised to protect the innocent!

Scotsmen & the Weather.

Norman — Tue, 03 Apr 2012 10:30:57 +0000

This one has been around for a while, but it’s still amusing. Other opinions are available …

40 degrees Fahrenheit-Californians shiver uncontrollably.
People in Scotland sunbathe.

35 degrees-Italian cars won’t start.
People in Scotland drive with the windows down.

20 degrees-Floridians wear coats, gloves, and wool hats.
People in Scotland throw on a T-shirt.

15 degrees-Californians begin to evacuate the state.
People in Scotland go swimming.

Zero degrees-New York landlords finally turn up the heat.
People in Scotland have the last bbq before it gets cold.

10 degrees below zero-People in Miami cease to exist.
People in Scotland lick flagpoles.

20 degrees below zero-Californians fly away to Mexico.
People in Scotland throw on a light jacket.

80 degrees below zero-Polar bears begin to evacuate the Arctic.
Scottish Boy Scouts postpone “Winter Survival” classes until it gets cold enough.

100 degrees below zero-Santa Claus abandons the North Pole.
People in Scotland pull down their ear flaps.

173 degrees below zero-Ethyl alcohol freezes.
People in Scotland get frustrated when they can’t thaw their kegs.

297 degrees below zero-Microbial life start to disappear.
Scottish cows complain of farmers with cold hands.

460 degrees below zero-ALL atomic motion stops.
People in Scotland start saying “chilly, you cauld an aw?”
(Translation: “It’s chilly, are you cold as well?”)

500 degrees below zero-Hell freezes over.
Scottish people support England in World Cup!!!!

Sudo on SLES Asking for Root Password?

Norman — Wed, 21 Mar 2012 12:42:41 +0000

It’s a post-installation configuration that hasn’t been done. The /etc/sudoers file has the following section in it:

# In the default (unconfigured) configuration, sudo asks for the root password.
# This allows use of an ordinary user account for administration of a freshly
# installed system. When configuring sudo, delete the two
# following lines:
Defaults targetpw   # ask for the password of the target user i.e. root
ALL ALL=(ALL) ALL   # WARNING! Only use this together with 'Defaults targetpw'!

The administrator needs to edit /etc/sudoers and comment out the final two lines above, or remove them altogether. Once done, individual user accounts can be added as follows:

oracle ALL=(ALL) ALL
norman ALL=(ALL) ALL

or groups can be given access as follows:

%dba ALL=(ALL) ALL
%oinstall ALL=(ALL) ALL

Substituting the appropriate levels of commands and privileges of course!

Oracle RMAN for Beginners – Part 7

Norman — Mon, 19 Mar 2012 21:16:37 +0000

In part 6 of this mini-series, I left you with a backed up database using RMAN to take hot backups. This episode looks at restoring and recovering from hot backups.

The joy of this is that most of the time you don’t need to have everyone off of the database twiddling their thumbs while you restore and recover, just anyone in those areas affected.

Restoration and recovery is full in that the database will be completely recovered right up to date after the restore and recovery is finished.

Recover the Entire Database

To recover the entire database you do actually need to have the databases mounted so the users will need to be offline. The steps involved are:

Make sure that the database is mounted, not open:

RMAN> shutdown

database closed
database dismounted
Oracle instance shut down

RMAN> startup mount

connected to target database (not started)
Oracle instance started
database mounted

Total System Global Area     768331776 bytes

Fixed Size                     2230360 bytes
Variable Size                213911464 bytes
Database Buffers             549453824 bytes
Redo Buffers                   2736128 bytes

Restore the database from a suitable dump. RMAN will choose the dump for you in order to reduce the amount of work required:

RMAN> restore database;

Starting restore at 2012/03/19 21:19:35
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=17 device type=DISK

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00002 to /srv/nffs/oradata/ant12/data/sysaux01.dbf
channel ORA_DISK_1: restoring datafile 00003 to /srv/nffs/oradata/ant12/data/undotbs01.dbf
...
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
Finished restore at 2012/03/19 21:24:02

Recover the database. This will use any available backups of archived logs, the currently online and possibly never backed up archived logs plus the online redo logfiles to bring the database right up to date. Any archived logs that are no longer online will be restored first – which you can see in the following if you looks carefully:

RMAN> recover database;

Starting recover at 2012/03/19 21:28:58
using channel ORA_DISK_1

starting media recovery

archived log for thread 1 with sequence 12 is already on disk as file /srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_12_7mwyxlk8_.arc
archived log for thread 1 with sequence 13 is already on disk as file /srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_13_7mwzxtxg_.arc
...
channel ORA_DISK_1: starting archived log restore to default destination
channel ORA_DISK_1: restoring archived log
archived log thread=1 sequence=8
channel ORA_DISK_1: reading from backup piece /srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_annnn_TAG20120217T160332_7mwylo3r_.bkp
...
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
...
archived log file name=/srv/nffs/flashback_area/ant12/NORM/archivelog/2012_03_19/o1_mf_1_9_7ph990ph_.arc thread=1 sequence=9
channel default: deleting archived log(s)
archived log file name=/srv/nffs/flashback_area/ant12/NORM/archivelog/2012_03_19/o1_mf_1_9_7ph990ph_.arc RECID=77 STAMP=778368544
archived log file name=/srv/nffs/flashback_area/ant12/NORM/archivelog/2012_03_19/o1_mf_1_10_7ph990r7_.arc thread=1 sequence=10
channel default: deleting archived log(s)
archived log file name=/srv/nffs/flashback_area/ant12/NORM/archivelog/2012_03_19/o1_mf_1_10_7ph990r7_.arc RECID=76 STAMP=778368544
channel ORA_DISK_1: starting archived log restore to default destination
channel ORA_DISK_1: restoring archived log
archived log thread=1 sequence=11
channel ORA_DISK_1: reading from backup piece /srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_annnn_TAG20120217T160922_7mwyxlvn_.bkp
channel ORA_DISK_1: piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_annnn_TAG20120217T160922_7mwyxlvn_.bkp tag=TAG20120217T160922
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
archived log file name=/srv/nffs/flashback_area/ant12/NORM/archivelog/2012_03_19/o1_mf_1_11_7ph996wn_.arc thread=1 sequence=11
channel default: deleting archived log(s)
archived log file name=/srv/nffs/flashback_area/ant12/NORM/archivelog/2012_03_19/o1_mf_1_11_7ph996wn_.arc RECID=78 STAMP=778368551
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_12_7mwyxlk8_.arc thread=1 sequence=12
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_13_7mwzxtxg_.arc thread=1 sequence=13
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_14_7mwzxx1n_.arc thread=1 sequence=14
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_15_7mwzxxkb_.arc thread=1 sequence=15
media recovery complete, elapsed time: 00:00:25
Finished recover at 2012/03/19 21:29:36

And finally, open the database:

RMAN> alter database open;

database opened

Recover Tablespaces

Recovering individual tablespaces is done with the database open. Only the tablespaces to be recovered need to be offline.

However, if the SYSTEM or UNDO tablespaces need to be recovered, the database will need to be mounted as you cannot restore and recover those with the database online – for pretty obvious reasons to be honest!

Note: You cannot recover a tablespace that has been dropped. In that situation, you must perform a point in time recovery to just before the tablespace was dropped. The control file doesn’t keep details of the dropped tablespace.

Attempting to recover a dropped tablespace will result in RMAN-20202 Tablespace not found in the recovery catalog errors.

You can, however, recover a tablespace where one or more of its datafiles have become corrupted or have been removed by nefarious means.

The following example shows the users tablespace being restored from a backup and recovered completely up to date.

First of all, in an SQL*Plus session, add an up to date record to a test table in the tablespace to be restored and recovered:

SQL> insert into test values (sysdate);

1 row created.

SQL> select * from test order by a desc;

A
-------------------
2012/03/19 20:58:18
2012/03/07 11:21:53
...

This change has not yet been archived so will be found in the online redo logs. The following is the RMAN recovery & restore process.

The first step is to take the affected tablespaces offline:

RMAN> sql 'alter tablespace users offline';

sql statement: alter tablespace users offline

The next step will restore the datafiles in this tablespace from a suitable dump. RMAN will choose the dump in order to reduce the amount of work it has to do to complete the restoration:

RMAN> restore tablespace users;

Starting restore at 2012/03/19 21:01:25
using channel ORA_DISK_1

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00006 to /srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: reading from backup piece /media/oracle_backups/ant12/2nn3ict8_1_1
channel ORA_DISK_1: piece handle=/media/oracle_backups/ant12/2nn3ict8_1_1 tag=TAG20120217T165152
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
Finished restore at 2012/03/19 21:01:26

With the dump of the datafile(s) restored, I next need to recover the tablespace to the current state:

RMAN> recover tablespace users;

Starting recover at 2012/03/19 21:01:32
using channel ORA_DISK_1

starting media recovery
media recovery complete, elapsed time: 00:00:01

Finished recover at 2012/03/19 21:01:33

Finally, bring the tablespace back online:

RMAN> sql 'alter tablespace users online';

sql statement: alter tablespace users online

And to be sure that I have indeed recovered the tablespace completely up to date, I re-ran the above query in my SQL*Plus session again:

SQL> select * from test order by a desc;

A
-------------------
2012/03/19 20:58:18
2012/03/07 11:21:53
...

If the SYSTEM or UNDO tablespace need restoring and recovery then the database has to be mounted, as follows:

RMAN> shutdown

database closed
database dismounted
Oracle instance shut down

RMAN> startup mount

connected to target database (not started)
Oracle instance started
database mounted

Total System Global Area     768331776 bytes

Fixed Size                     2230360 bytes
Variable Size                213911464 bytes
Database Buffers             549453824 bytes
Redo Buffers                   2736128 bytes

RMAN> restore tablespace system;

Starting restore at 2012/03/19 21:14:34
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=17 device type=DISK

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00001 to /srv/nffs/oradata/ant12/data/system01.dbf
channel ORA_DISK_1: reading from backup piece /srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_nnndf_TAG20120217T161210_7mwz2tyv_.bkp
channel ORA_DISK_1: piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_nnndf_TAG20120217T161210_7mwz2tyv_.bkp tag=TAG20120217T161210
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:25
Finished restore at 2012/03/19 21:15:00

RMAN> recover tablespace system;

Starting recover at 2012/03/19 21:15:07
using channel ORA_DISK_1

starting media recovery

archived log for thread 1 with sequence 13 is already on disk as file /srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_13_7mwzxtxg_.arc
archived log for thread 1 with sequence 14 is already on disk as file /srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_14_7mwzxx1n_.arc
archived log for thread 1 with sequence 15 is already on disk as file /srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_15_7mwzxxkb_.arc
archived log for thread 1 with sequence 16 is already on disk as file /srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_16_7mwzxzvm_.arc
archived log for thread 1 with sequence 17 is already on disk as file /srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_17_7mwzy01m_.arc
archived log for thread 1 with sequence 18 is already on disk as file /srv/nffs/flashback_area/ant12/NORM/archivelog/2012_03_19/o1_mf_1_18_7ph695lw_.arc
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_13_7mwzxtxg_.arc thread=1 sequence=13
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_14_7mwzxx1n_.arc thread=1 sequence=14
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_15_7mwzxxkb_.arc thread=1 sequence=15
media recovery complete, elapsed time: 00:00:02
Finished recover at 2012/03/19 21:15:10

RMAN> alter database open;

database opened

Recover Datafiles

Recovering datafiles instead of a complete tablespace could save you a lot of downtime for the affected users. As ever, the affected datafile(s) need to be offline in order to be recovered. The remainder of the process is as simple as restoring and recovering a tablespace so the following demonstration of a datafile recovery needs little comment, however, as before, if SYSTEM or UNDO are affected, the database needs to be mounted, not open.

RMAN> sql 'alter database datafile 5 offline';

sql statement: alter database datafile 5 offline

RMAN> restore datafile 5;

Starting restore at 2012/03/19 21:40:50
using channel ORA_DISK_1

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00005 to /srv/nffs/oradata/ant12/data/tools01.dbf
channel ORA_DISK_1: reading from backup piece /srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_nnndf_TAG20120217T161511_7mwz8hvx_.bkp
channel ORA_DISK_1: piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_nnndf_TAG20120217T161511_7mwz8hvx_.bkp tag=TAG20120217T161511
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
Finished restore at 2012/03/19 21:40:52

RMAN> recover datafile 5;

Starting recover at 2012/03/19 21:40:57
using channel ORA_DISK_1

starting media recovery

archived log for thread 1 with sequence 13 is already on disk as file /srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_13_7mwzxtxg_.arc
...
media recovery complete, elapsed time: 00:00:00
Finished recover at 2012/03/19 21:40:58

RMAN> sql 'alter database datafile 5 online';

sql statement: alter database datafile 5 online

Of course, the above assumes you know the datafile number, the report schema command helps here. You can always run the restore and recovery using the full filename in single quotes, but I find the data file number method to be easier and less prone to my abysmal typing skills!

If you have multiple files to restore and recover, separate them with a comma in the normal fashion:

RMAN> sql 'alter database datafile 5, 6 offline';
...
RMAN> recover datafile 5, 6;
...
RMAN> restore datafile 5, 6;
...
RMAN> sql 'alter database datafile 5, 6 online';

Recover Individual Blocks

If you can save time by restoring and recovering just a datafile or two rather than a complete tablespace, then how about restoring and recovering a block or two, that’s will save even more time surely?

RMAN can help you out here as well, and you don’t need the datafile(s) to be offline at the time.

RMAN> recover datafile 5 block 24;

Starting recover at 2012/03/19 21:49:47
using channel ORA_DISK_1

starting media recovery
media recovery complete, elapsed time: 00:00:00

Finished recover at 2012/03/19 21:49:47

In this case, even the SYSTEM and UNDO tablespaces can be recovered while online. You should note that there is no need to restore the data block first, just run the recover command.

See the RMAN manual for many other formats of this useful command.

Rename an Oracle 10g or 11g Database

Norman — Wed, 07 Mar 2012 16:04:28 +0000

Due to a document not being supplied to me recently, I built a few 11g databases with the wrong SID. I needed to change them all. I found a few web pages, Oracle and others, on using the nid utility to do just that, however, they were incomplete. The full process is described here.

The following has been tested fully on an 11.2.0.2 Oracle database. The nid utility used also exists in Oracle 10.2.0.5 and possibly other versions that I do not have access to at the moment.

Backup the database or ensure a backup exists in RMAN. This shouldn’t be necessary but you are a DBA right? And chances we do not take!
create pfile='/home/oracle/initnew_sid.ora' from spfile; this ensures that the pfile you are about to edit contains all the current settings. It is saved in a non-standard location for safety.
Shutdown the database cleanly. Use shutdown or shutdown immediate.
Startup mount the database.
Check the open_cursors parameter. If you have a lot of data files you might need to increase this setting temporarily:
```
alter system set open_cursors=1500 scope=memory;
```
In a shell session run the following nid command:
```
nid target=/ dbname=new_sid setname=y logfile=new_name.log
```
The logfile created will show details of what just happened and should be checked for errors. As far as I am aware, but don’t trust me on this, errors will not change the database at all.

The instance will also have been shut down at the end of the nid command.
Edit /etc/oratab to change the old sid to the new one.
Edit tnsnames.ora to do likewise. Also applies to OID, LDAP, whatever you use for alias resolution.
Stop the appropriate listener, edit listener.ora and restart the listener. If the listener in question serves other databases, just edit the listener.ora file and run lsnrctl reload listener_name.
Copy the newly created /home/oracle/initnew_sid.ora to $ORACLE_HOME/dbs/initnew_sid.ora then edit the new file and change the db_name parameter.
You may, if desired, delete the old files $ORACLE_HOME/dbs/initold_sid.ora and $ORACLE_HOME/dbs/spfileold_sid.ora as they are no longer needed. (Unless you plan on renaming the database back again of course!)
Export ORACLE_SID as the new sid. ORACLE_HOME will remain the same as before.
Run the orapwd command to create a new password file if required.
Startup the database.
create spfile='oracle_home/dbs/spfilenew_sid.ora' from pfile; Obviously substituting the correct values for new_sid and oracle_home.
Shutdown and restart the database to use the new spfile.

Check it all just worked:

select name from v$database;
show parameter db_name

That’s the simple bit and in theory is all you need to do. However, be aware that any data files, control files, recovery file destinations etc still have their old format names. You may end up with a database called fred located in /srv/barney/oradata for example. If you wish to go down the route of renaming everything to suit the database name, see part 2 of this discussion, here.

As the control files are not overwritten by this process, and because the DBID doesn’t change, all your RMAN backup information is safe and even after the database has been renamed, you can use an old backup to restore and recover the database. I’ve tested this on a tablespace recovery to be sure.

Ok, when I say your backups are safe, they are, but if you try to restore the controlfile from an old pre-rename dump, it will restore with no errors. However, when you attempt to mount the database you will be informed that the name in the controlfile doesn’t match the database. Edit the restored controlfile(s) to change the name and continue.

I’m pretty sure that because the DBID doesn’t change, an RMAN catalogue will not lose any backup details either. On my test system, I don’t yet have a catalogue to play with, so best you test on an expendable database first. Just saying!

Online and backed up archived logs as well as the online redo logs are used quite happily to apply any required REDO.

It just works!

Oracle RMAN for Beginners – Part 6

Norman — Fri, 17 Feb 2012 16:57:53 +0000

In the past few instalments, I’ve looked mainly at databases running in NOARCHIVELOG mode. It’s probably not the best way to run a production system but it is valid to do so.

For the safest systems and the ability to recover from numerous disasters without losing nay data, the database should be run in ARCHIVELOG mode.

The remainder of the mini-series concentrates on the things you can do with RMAN when the database is running in ARCHIVELOG mode.

Hot Backups

There’s nothing to taking a hot backup that I haven’t already shown you. Taking a backup of the full database in hot (or online) mode is exactly the same as taking a cold (offline) backup.

One difference is that you must backup the archived logs in addition to the database.

Another difference is that you can take a hot backup while the database is open, although you can also take one while the database is mounted.

In the following examples, I have set my archivelog deletion policy as follows:

RMAN> configure archivelog deletion policy backed up 2 times to disk;

new RMAN configuration parameters:
CONFIGURE ARCHIVELOG DELETION POLICY TO BACKED UP 2 TIMES TO DISK;
new RMAN configuration parameters are successfully stored

Backup the Database and Archived Logs

To backup the database, including controlfile and spfile if autobackup is configured – and the archived logs, all I need to do is as follows:

RMAN> run {
2> backup full database;
3> backup archivelog all delete input;
4> }

An extract of the results of running the above is this:

Starting backup at 2012/02/17 15:07:46
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=45 device type=DISK
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00010 name=/srv/nffs/oradata/ant12/data/NLWLDELFTFEWSModDat01_01.dbf
...
input datafile file number=00006 name=/srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: starting piece 1 at 2012/02/17 15:07:47
channel ORA_DISK_1: finished piece 1 at 2012/02/17 15:08:32
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_nnndf_TAG20120217T150747_7mwvb3sy_.bkp tag=TAG20120217T150747 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:45
Finished backup at 2012/02/17 15:08:32

Starting backup at 2012/02/17 15:08:34
current log archived
using channel ORA_DISK_1
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=1 RECID=57 STAMP=775494253
input archived log thread=1 sequence=2 RECID=58 STAMP=775494254
input archived log thread=1 sequence=3 RECID=59 STAMP=775494255
input archived log thread=1 sequence=4 RECID=60 STAMP=775494258
input archived log thread=1 sequence=5 RECID=61 STAMP=775494274
input archived log thread=1 sequence=6 RECID=62 STAMP=775494514
channel ORA_DISK_1: starting piece 1 at 2012/02/17 15:08:35
channel ORA_DISK_1: finished piece 1 at 2012/02/17 15:08:38
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_annnn_TAG20120217T150834_7mwvcm3s_.bkp tag=TAG20120217T150834 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:03

The archived logs are backed up to the FRA, as usual, and then deleted from their normal position in the FRA.This helps keep the FRA tidy of archived logs which are no longer necessary as enough backups have been taken.

Setting the archived log deletion policy sensibly means that you should never be without the logs you need to recover a restored database, either in full or up to a specific point in time, scn or leg sequence number.

You may note, if you looked closely, that one of the first things RMAN did as part of the archived log backup, was to archive the current online redo log. This means that you will have all the archived logs necessary since the start of the backup (of the database) available in the event of a recovery being required.

As this is the first time my archived logs have been backed up, the attempt to delete them has not been successful, as the following extract shows:

channel ORA_DISK_1: deleting archived log(s)
RMAN-08138: WARNING: archived log not deleted - must create more backups
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_1_7mwv3ddf_.arc thread=1 sequence=1
...
RMAN-08138: WARNING: archived log not deleted - must create more backups
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_6_7mwvclf7_.arc thread=1 sequence=6
Finished backup at 2012/02/17 15:08:38

These warnings do not affect the backup. The archived logs in the FRA have been successfully backed up (to the FRA) but because the deletion policy states that they can only be deleted, from the FRA, when they have already been backed up twice to disc, then they must remain on disc in the FRA.

Finally, the controlfile and spfile are automatically backed up:

Starting Control File and SPFILE Autobackup at 2012/02/17 15:08:38
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_17/o1_mf_s_775494518_7mwvcpqn_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/17 15:08:41

It may be simpler to run the following command instead:

RMAN> backup full database plus archivelog delete input;

Looking at the log from the above command shows that the following steps take place:

The current redo log is archived.
The archived logs are (all) backed up.
The deletion policy for archived logs is applied.
The database is backed up.
The (new) current redo log is archived.
The newly archived log is backed up on its own.
The controlfile and spfile are backed up – if configured to do so.

An extract from the log, showing the above steps, is as follows:

Starting backup at 2012/02/17 15:37:19
current log archived
using channel ORA_DISK_1
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=1 RECID=57 STAMP=775494253
...
input archived log thread=1 sequence=7 RECID=63 STAMP=775496239
channel ORA_DISK_1: starting piece 1 at 2012/02/17 15:37:20
channel ORA_DISK_1: finished piece 1 at 2012/02/17 15:37:23
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_annnn_TAG20120217T153720_7mwx1j93_.bkp tag=TAG20120217T153720 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:03

channel ORA_DISK_1: deleting archived log(s)
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_1_7mwv3ddf_.arc RECID=57 STAMP=775494253
...
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_6_7mwvclf7_.arc RECID=62 STAMP=775494514
RMAN-08138: WARNING: archived log not deleted - must create more backups
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_7_7mwx1hj5_.arc thread=1 sequence=7
Finished backup at 2012/02/17 15:37:23

Starting backup at 2012/02/17 15:37:23
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00010 name=/srv/nffs/oradata/ant12/data/NLWLDELFTFEWSModDat01_01.dbf
...
input datafile file number=00006 name=/srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: starting piece 1 at 2012/02/17 15:37:24
channel ORA_DISK_1: finished piece 1 at 2012/02/17 15:38:09
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_nnndf_TAG20120217T153723_7mwx1n4f_.bkp tag=TAG20120217T153723 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:45
Finished backup at 2012/02/17 15:38:09

Starting backup at 2012/02/17 15:38:09
current log archived
using channel ORA_DISK_1
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=8 RECID=64 STAMP=775496289
channel ORA_DISK_1: starting piece 1 at 2012/02/17 15:38:09
channel ORA_DISK_1: finished piece 1 at 2012/02/17 15:38:10
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_annnn_TAG20120217T153809_7mwx31qd_.bkp tag=TAG20120217T153809 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01

channel ORA_DISK_1: deleting archived log(s)
RMAN-08138: WARNING: archived log not deleted - must create more backups
archived log file name=/srv/nffs/flashback_area/ant12/ANT12/archivelog/2012_02_17/o1_mf_1_8_7mwx31h2_.arc thread=1 sequence=8
Finished backup at 2012/02/17 15:38:10

Starting Control File and SPFILE Autobackup at 2012/02/17 15:38:10
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_17/o1_mf_s_775496290_7mwx33bn_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/17 15:38:13

Backup Selected Tablespaces

One of the joys of running a database in ARCHIVELOG mode is the fact that it is possible to backup and recover individual tablespaces.

RMAN> backup tablespace users;

Starting backup at 2012/02/17 15:59:07
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00006 name=/srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: starting piece 1 at 2012/02/17 15:59:07
channel ORA_DISK_1: finished piece 1 at 2012/02/17 15:59:08
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_nnndf_TAG20120217T155907_7mwybcbd_.bkp tag=TAG20120217T155907 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 2012/02/17 15:59:08

Starting Control File and SPFILE Autobackup at 2012/02/17 15:59:08
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_17/o1_mf_s_775497548_7mwybdnt_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/17 15:59:09

Including the archived logs at the same time, is possible too, by running the backup tablespace tools plus archivelog delete input; command. It isn’t mandatory to delete the archived logs as part of the command, but it helps keep the FRA tidy.

Backup Selected Data Files

With ARCHIVELOG mode enabled, you can even backup at the data file level.

Datafailes may be specified by id or my full name. The following backs up the SYSTEM tablespace’s single datafile.

RMAN> report schema;

Report of database schema for database with db_unique_name ANT12

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    600      SYSTEM               ***     /srv/nffs/oradata/ant12/data/system01.dbf
...
12   500      XDB                  ***     /srv/nffs/oradata/ant12/data/xdb01.dbf


RMAN> backup datafile 1;

Starting backup at 2012/02/17 16:12:10
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00001 name=/srv/nffs/oradata/ant12/data/system01.dbf
channel ORA_DISK_1: starting piece 1 at 2012/02/17 16:12:10
channel ORA_DISK_1: finished piece 1 at 2012/02/17 16:12:26
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_nnndf_TAG20120217T161210_7mwz2tyv_.bkp tag=TAG20120217T161210 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:16
Finished backup at 2012/02/17 16:12:26

Starting Control File and SPFILE Autobackup at 2012/02/17 16:12:26
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_17/o1_mf_s_775498346_7mwz3bdl_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/17 16:12:27

You can also backup multiple data files together by specifying a comma separated list of id numbers or names:

RMAN> backup datafile
2> '/srv/nffs/oradata/ant12/data/tools01.dbf',
3> '/srv/nffs/oradata/ant12/data/users01.dbf';

Starting backup at 2012/02/17 16:15:11
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00005 name=/srv/nffs/oradata/ant12/data/tools01.dbf
input datafile file number=00006 name=/srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: starting piece 1 at 2012/02/17 16:15:11
channel ORA_DISK_1: finished piece 1 at 2012/02/17 16:15:12
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_nnndf_TAG20120217T161511_7mwz8hvx_.bkp tag=TAG20120217T161511 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 2012/02/17 16:15:12

Starting Control File and SPFILE Autobackup at 2012/02/17 16:15:12
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_17/o1_mf_s_775498513_7mwz8k80_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/17 16:15:16

The above is equivalent to the command backup datafile 5,6; in the case of my database.

Backup Archived Logs

You can take a backup of the archived logs at any time you wish simply by running the backup archivelog all; command as demonstrated at the beginning of this instalment.

It is possible to backup only a selection of archived logs, for example, a single log file:

RMAN> backup archivelog sequence 15;

Starting backup at 2012/02/17 16:27:15
using channel ORA_DISK_1
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=15 RECID=71 STAMP=775499197
channel ORA_DISK_1: starting piece 1 at 2012/02/17 16:27:15
channel ORA_DISK_1: finished piece 1 at 2012/02/17 16:27:16
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_annnn_TAG20120217T162715_7mwzz3gk_.bkp tag=TAG20120217T162715 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 2012/02/17 16:27:16

Starting Control File and SPFILE Autobackup at 2012/02/17 16:27:16
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_17/o1_mf_s_775499236_7mwzz4w2_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/17 16:27:17

Or, if required, a number of them:

RMAN> backup archivelog sequence between 16 and 19;

Starting backup at 2012/02/17 16:28:15
using channel ORA_DISK_1
channel ORA_DISK_1: starting archived log backup set
channel ORA_DISK_1: specifying archived log(s) in backup set
input archived log thread=1 sequence=16 RECID=72 STAMP=775499199
input archived log thread=1 sequence=17 RECID=73 STAMP=775499200
channel ORA_DISK_1: starting piece 1 at 2012/02/17 16:28:16
channel ORA_DISK_1: finished piece 1 at 2012/02/17 16:28:17
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_17/o1_mf_annnn_TAG20120217T162815_7mx01041_.bkp tag=TAG20120217T162815 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 2012/02/17 16:28:17

Starting Control File and SPFILE Autobackup at 2012/02/17 16:28:17
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_17/o1_mf_s_775499297_7mx011ll_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/17 16:28:18

You may also specify the logs to be backed up using times. For example, the command backup archivelog from time 'sysdate-7' until time 'sysdate-1'; will backup those archived logs, still on disc in the FRA, that were created within the time period specified.

And finally, for archived logs, if they have been stored in a section of the file system which is outside the FRA, you can still back those up.

If the archived logs are in the location ‘/srv/nffs/oradata/arch’ then you back those up (to the FRA) by running the command backup archivelog like '/srv/nffs/oradata/arch%';

Specifying a New Destination for Backups

In all of the preceding examples, the backups have been created in the FRA. What do you do if yo want to create a backup somewhere else?

The format parameter is how:

RMAN> backup tablespace users format='/media/oracle_backups/ant12/%U';

Starting backup at 2012/02/17 16:51:52
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00006 name=/srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: starting piece 1 at 2012/02/17 16:51:52
channel ORA_DISK_1: finished piece 1 at 2012/02/17 16:51:53
piece handle=/media/oracle_backups/ant12/2nn3ict8_1_1 tag=TAG20120217T165152 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 2012/02/17 16:51:53

Starting Control File and SPFILE Autobackup at 2012/02/17 16:51:54
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_17/o1_mf_s_775500714_7mx1fbdd_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/17 16:51:55

It can bee seen from the piece handle that the backup has indeed been sent to the non-FRA disc area.

The format parameter can be used for all types of backups, not just for tablespaces as in this example.

Oracle RMAN for Beginners – Part 5

Norman — Mon, 13 Feb 2012 16:00:42 +0000

Previously in this mini-series, I managed to dump and recover a database from a cold backup. Before I move on to similar practices with hot backups, a few terms and observations may well be in order.

Terms and Conditions

A restore is the action of copying files back from a backup, prior to recovery.

Recovery is carried out after restoring the files and allows the database to be recovered in full or to a specific point in time, using backups of archived logs, and/or the current archived logs – which may not have been backed up yet.

A full recovery restores and recovers the database to an up to date point. No data will be lost or missing after the recovery.

A partial recovery restores and recovers the database to a point in time in the past. Data may be lost after the recovery has completed.

A consistent backup is a cold backup. I have been using these types of backup since the beginning of this series of articles.

A consistent backup is one which, when recovered, leaves the database ready to use with no further recovery required.

An inconsistent backup is a hot backup. We will be looking at these backups next as they are the ones most likely to be in popular use.

An inconsistent backup has to be restored and then rolled forward (recovered) to some specific point in time before the database can be opened for use.

An available backup is a backup, consistent or otherwise, which is available for use in recovering a database.

An unavailable backup is a backup, consistent or otherwise, which is deemed to be unavailable for use in recovering a database. In normal conditions, this is a forced action by the DBA who marks the backup as unavailable in order to prevent RMAN using it in a restoration

An expired backup is a backup which is registered in the catalogue and/or controlfile, but which is not found on disc or tape.

An obsolete backup is a backup for which the files remain on disc or tape, but the configured retention period has caused the backup to become no longer required.

A full backup is a backup of the whole database.

An incremental backup is a backup which is not a full backup, but only contains blocks which have changed since the previous level 0 or level 1 incremental backup depending on whether the backup is cumulative or differential.

Retention and Deletion Policy

Oracle advise four (main) things when using RMAN to backup and recover your databases:

Use a flashback recovery area
Use a backup retention policy
Use an archive log deletion policy
Use a recovery catalogue.

The use of the FRA has been covered already so no more need be said about that.

I will be discussing a recover catalogue later in this mini-series. For now, I’m sticking with the use only of the control file to record database backup details.

Retention Policy

You may configure a retention policy for your backups to ensure that they remain available for as long as you deem necessary. You may define the retention as a recovery window – the number of days a backups should remain available for – or as a number of copies.

RMAN’s default is a retention policy of one copy. This means that as each new database backup is created, the previous one becomes obsolete.

The method you use will depend on how your organisation likes to work. If the rules are that all backups should be kept for a fortnight then the command configure retention policy to recovery window of 14 days will cover that rule.

If, on the other hand, the rules demand that four copies of all backups should be retained then the command required will be configure retention policy to redundancy 4.

Whichever way that you configure your retention policy, backups that fall outside of your configured policy will automatically be flagged as obsolete by RMAN.

Note: RMAN may well hold on to a backup beyond its retention period if any of the files within the backup are still required to be able to restore the database.

With backup optimisation configured on, files that have not changed since the previous backup will not be backed up. This speeds up the backup process but does mean that more backups may be required to allow everything to be restored.

Archive log Deletion Policy

The archive log deletion policy is a similar system to specify which archived logs are eligible for deleting from the FRA to recover space.

RMAN will delete those eligible logs as and when it needs to recover space in the FRA for further backups, or to avoid exceeding the setting of the DB_RECOVERY_FILE_DEST_SIZE for the database.

If you have mirrored copies of your archived logs stored outside the FRA, those will not be deleted automatically by RMAN, it is your responsibility to delete them.

The deletion policy also controls whether or not archived logs are deleted from the FRA when the backup archivelog all delete input command is executed. Any archived logs that have not yet been backed up enough times, or have been backed up but for too few days, will not be deleted. You can force the deletion though.

Examples of setting up the deletion policy are configure archivelog deletion policy to applied on all standby or configure archivelog deletion policy to backed up 5 times to sbt etc.

The default setting is none which means that archived logs are never deleted automatically from the FRA.

Clone Oracle Home Easily

Norman — Thu, 09 Feb 2012 08:23:52 +0000

Frits Hoogland’s post on how to clone your Oracle Home rather than installing from scratch. Nice!

Oracle RMAN for Beginners – Part 4

Norman — Mon, 06 Feb 2012 21:19:22 +0000

So far I have managed to dump and recover a database running in ARCHIVELOG mode. That is the most sensible mode for a production database and will be the case for the rest of this small RMAN guide. However, what if your databases are not running in ARCHIVELOG mode? What can RMAN do for you?

Running a Database in NOARCHIVELOG Mode

The database is shutdown at the moment, so we will MOUNT it using RMAN and create a brand new full cold backup in the FRA.

RMAN> connect target /
connected to target database (not started)

RMAN> startup mount

Oracle instance started
database mounted
...

RMAN> backup full database;

Starting backup at 2012/02/06 20:47:41
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=18 device type=DISK
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00010 name=/srv/nffs/oradata/ant12/data/NLWLDELFTFEWSModDat01_01.dbf
...
input datafile file number=00006 name=/srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: starting piece 1 at 2012/02/06 20:47:43
channel ORA_DISK_1: finished piece 1 at 2012/02/06 20:48:28
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_06/o1_mf_nnndf_TAG20120206T204742_7m0h3hmw_.bkp tag=TAG20120206T204742 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:45
Finished backup at 2012/02/06 20:48:28

Starting Control File and SPFILE Autobackup at 2012/02/06 20:48:28
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_06/o1_mf_s_774563524_7m0h4xng_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/06 20:48:31

RMAN> startup;
database is already started
database opened

Again, we will trash the database by renaming a data file, after doing some work in a SQL*Plus session. The session to attempt a recovery using RMAN went like this:

RMAN> startup

connected to target database (not started)
Oracle instance started
database mounted
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of startup command at 02/06/2012 21:00:45
ORA-01157: cannot identify/lock data file 6 - see DBWR trace file
ORA-01110: data file 6: '/srv/nffs/oradata/ant12/data/users01.dbf'

I know it’s the users tablespace that’s affected, so I will attempt to restore and recover just that:

RMAN> restore tablespace users;

Starting restore at 2012/02/06 21:01:21
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=18 device type=DISK

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00006 to /srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: reading from backup piece /srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_06/o1_mf_nnndf_TAG20120206T204742_7m0h3hmw_.bkp
channel ORA_DISK_1: piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_06/o1_mf_nnndf_TAG20120206T204742_7m0h3hmw_.bkp tag=TAG20120206T204742
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
Finished restore at 2012/02/06 21:01:23

It looks like RMAN allows a single tablespace to be restored, will it recover?

RMAN> recover tablespace users;

Starting recover at 2012/02/06 21:01:44
using channel ORA_DISK_1

starting media recovery
...
RMAN-08187: WARNING: media recovery until SCN 855319 complete
Finished recover at 2012/02/06 21:01:46

That warning doesn’t look too healthy. I next attempt to open the database:

RMAN> startup;

database is already started
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of startup command at 02/06/2012 21:02:09
ORA-01113: file 6 needs media recovery
ORA-01110: data file 6: '/srv/nffs/oradata/ant12/data/users01.dbf'

So, that’s it, with a database running in NOARCHIVELOG mode, you lose data if the changes that you are trying to recover have aged out of the online redo logs. Had they still been there I would have been able to recover the tablespace and open the database, as it is, I now need to do a full restore to get a consistent database back up and running.

RMAN> restore database;

Starting restore at 2012/02/06 21:08:11
using channel ORA_DISK_1

skipping datafile 6; already restored to file /srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00001 to /srv/nffs/oradata/ant12/data/system01.dbf
...
channel ORA_DISK_1: restoring datafile 00012 to /srv/nffs/oradata/ant12/data/xdb01.dbf
channel ORA_DISK_1: reading from backup piece /srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_06/o1_mf_nnndf_TAG20120206T204742_7m0h3hmw_.bkp
channel ORA_DISK_1: piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_06/o1_mf_nnndf_TAG20120206T204742_7m0h3hmw_.bkp tag=TAG20120206T204742
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:03:56
Finished restore at 2012/02/06 21:12:08

RMAN> sql "alter database open resetlogs";

sql statement: alter database open resetlogs

RMAN>

That’s it then, the database has been restored and opened.

One other thing, RMAN is smart enough to notice that my unsuccessful attempt to restore just the users tablespace meant that it didn’t have to restore it again. That saved a little time.

The moral to this little exercise is simple. Don’t run your databases in NOARCHIVELOG mode because you will lose data. In addition, you must always have downtime to backup the database. If you attempt to backup the database while it is open, you will see the following RMAN error message:

RMAN> backup full database;

Starting backup at 2012/02/07 07:19:35
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=18 device type=DISK
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03009: failure of backup command on ORA_DISK_1 channel at 02/07/2012 07:19:36
ORA-19602: cannot backup or copy active file in NOARCHIVELOG mode

Recovery From a Total Loss

Even in NOARCHIVEMODE and without using a catalogue, it is possible to recover a database when the controlfiles are lost or unusable. However, this is only possible, according to the manual, from a controlfile autobackup. You also must have the database unique identifier (DBID) to hand as well as the configured format of the controlfile autobackup files.

In the following example, I made sure that I had a backup of the database, the DBID (which RMAN helpfully displays when you connect to the target at the RMAN> prompt) and my controlfile autobackup format is the default, so I didn’t have to worry about that.

I shut down the database and deleted all the files. That represents a total failure scenario. The database is running in NOARCHIVELOG by the way. Let’s recover it.

RMAN> startup nomount

connected to target database (not started)
Oracle instance started
...

The database cannot be mounted as we have no controlfiles.

RMAN> set DBID=2799264292
executing command: SET DBID

This allows RMAN to try and find the controlfile autobackup.

RMAN> restore controlfile from autobackup;

Starting restore at 2012/02/07 08:21:39
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=19 device type=DISK

recovery area destination: /srv/nffs/flashback_area/ant12
database name (or database unique name) used for search: ANT12
channel ORA_DISK_1: AUTOBACKUP /srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_07/o1_mf_s_774602851_7m1noocb_.bkp found in the recovery area
channel ORA_DISK_1: looking for AUTOBACKUP on day: 20120207
channel ORA_DISK_1: restoring control file from AUTOBACKUP /srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_07/o1_mf_s_774602851_7m1noocb_.bkp
channel ORA_DISK_1: control file restore from AUTOBACKUP complete
output file name=/srv/nffs/oradata/ant12/ctrl/control01.ctl
output file name=/srv/nffs/flashback_area/ant12/ctrl/control02.ctl
output file name=/srv/nffs/oradata/ant12/ctrl/control03.ctl
Finished restore at 2012/02/07 08:21:40

So far so good. I have my controlfiles back. As I was using the default autobackup format, I didn’t have to set the format, however, if I had changed the format at some point, I need to tell RMAN what it is. In that case, the above restore would look like the following:

RMAN> set DBID=2799264292
executing command: SET DBID

RMAN> run {
2> set controlfile autobackup format for device type disk to 'your format here';
3> restore controlfile from autobackup;
4> }

The end result would be, hopefully, the restoration of the latest controlfile backup.

The controlfiles are back, but I’m are still without the data files.

Note: any time that you have to restore the controlfile as part of a restore means that you must open the database with the resetlogs option.

RMAN alter database mount;
database mounted

RMAN> restore database;
Starting restore at 2012/02/07 08:40:20
Starting implicit crosscheck backup at 2012/02/07 08:40:20
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=19 device type=DISK
Crosschecked 1 objects
Finished implicit crosscheck backup at 2012/02/07 08:40:21

Starting implicit crosscheck copy at 2012/02/07 08:40:21
using channel ORA_DISK_1
Crosschecked 17 objects
Finished implicit crosscheck copy at 2012/02/07 08:40:22

searching for all files in the recovery area
cataloging files...
cataloging done

List of Cataloged Files
=======================
File Name: /srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_07/o1_mf_s_774602851_7m1noocb_.bkp

using channel ORA_DISK_1

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00001 to /srv/nffs/oradata/ant12/data/system01.dbf
...
channel ORA_DISK_1: restoring datafile 00012 to /srv/nffs/oradata/ant12/data/xdb01.dbf
channel ORA_DISK_1: reading from backup piece /srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_07/o1_mf_nnndf_TAG20120207T072806_7m1nn6rq_.bkp
channel ORA_DISK_1: piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_07/o1_mf_nnndf_TAG20120207T072806_7m1nn6rq_.bkp tag=TAG20120207T072806
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:03:38
Finished restore at 2012/02/07 08:44:03

RMAN> alter database open resetlogs;
database opened

Now that’s what I call magic!

Oracle RMAN for Beginners – Part 3

Norman — Mon, 06 Feb 2012 16:49:00 +0000

At the end of Oracle RMAN for Beginners – Part 2 I had created a few backups of the ant12 database. In this part, I explore how to trash a database and recover from that trashing using the cold backups taken.

Bear in mind that any work done after the cold backup was taken will be lost. With a cold backup, you cannot roll forward to re-apply archived logs etc because when the backup was taken, the database was in a consistent state.

Note: A cold backup, taken of a database running in archivelog mode may still require archived logs to be present to restore the backup. Document 337450.1 on My Oracle Support has the details. This affects databases running RMAN version 9.2.0.1 through 10.2.0.5.

This could explain why I’m not seeing it as I’m restoring an 11.2 RMAN backup.

Thanks to David Farkough (@Farkough) for this information.

Cold Recovery – Backupset Type

In order to restore a database I first need a broken database. To see what files I need to destroy, I can use the RMAN command report schema as follows:

RMAN> report schema;

Report of database schema for database with db_unique_name ANT12

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    600      SYSTEM               ***     /srv/nffs/oradata/ant12/data/system01.dbf
2    500      SYSAUX               ***     /srv/nffs/oradata/ant12/data/sysaux01.dbf
3    512      UNDOTBS1             ***     /srv/nffs/oradata/ant12/data/undotbs01.dbf
4    700      PERFSTAT             ***     /srv/nffs/oradata/ant12/data/perfstat01_01.dbf
5    10       TOOLS                ***     /srv/nffs/oradata/ant12/data/tools01.dbf
6    10       USERS                ***     /srv/nffs/oradata/ant12/data/users01.dbf
7    300      AUDIT01              ***     /srv/nffs/oradata/ant12/data/audit01_01.dbf
8    1024     NLWLDELFTFEWSDAT01   ***     /srv/nffs/oradata/ant12/data/NLWLDELFTFEWSMCDat01_01.dbf
9    350      NLWLDELFTFEWSIDX01   ***     /srv/nffs/oradata/ant12/data/NLWLDELFTFEWSMCIdx01_01.dbf
10   3224     NLWLDELFTFEWSLOB01   ***     /srv/nffs/oradata/ant12/data/NLWLDELFTFEWSModDat01_01.dbf
11   128      UTILITY01            ***     /srv/nffs/oradata/ant12/data/utility01_01.dbf
12   500      XDB                  ***     /srv/nffs/oradata/ant12/data/xdb01.dbf

List of Temporary Files
=======================
File Size(MB) Tablespace           Maxsize(MB) Tempfile Name
---- -------- -------------------- ----------- --------------------
1    200      TEMP                 200         /srv/nffs/oradata/ant12/data/temp01.dbf

I shall start simple, and destroy the TOOLS tablespace by deleting the file.

RMAN> shutdown;
database dismounted
Oracle instance shut down

RMAN> host "mv /srv/nffs/oradata/ant12/data/tools01.dbf /srv/nffs/oradata/ant12/data/tools01.dbf.old";
host command complete

RMAN> startup
connected to target database (not started)
Oracle instance started
database mounted
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of startup command at 02/06/2012 16:14:41
ORA-01157: cannot identify/lock data file 5 - see DBWR trace file
ORA-01110: data file 5: '/srv/nffs/oradata/ant12/data/tools01.dbf'

You will note that the database didn’t open, but is stuck in a MOUNTed state. This is required to carry out a recovery. Lets restore the cold backup.

RMAN> connect target /
connected to target database: ANT12 (DBID=2799264292, not open)

RMAN> restore database;

Starting restore at 2012/02/06 16:15:49
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=18 device type=DISK
...
skipping datafile 1; already restored to file /srv/nffs/oradata/ant12/data/system01.dbf
skipping datafile 2; already restored to file /srv/nffs/oradata/ant12/data/sysaux01.dbf
skipping datafile 3; already restored to file /srv/nffs/oradata/ant12/data/undotbs01.dbf
skipping datafile 4; already restored to file /srv/nffs/oradata/ant12/data/perfstat01_01.dbf
skipping datafile 6; already restored to file /srv/nffs/oradata/ant12/data/users01.dbf
skipping datafile 7; already restored to file /srv/nffs/oradata/ant12/data/audit01_01.dbf
skipping datafile 8; already restored to file /srv/nffs/oradata/ant12/data/NLWLDELFTFEWSMCDat01_01.dbf
skipping datafile 9; already restored to file /srv/nffs/oradata/ant12/data/NLWLDELFTFEWSMCIdx01_01.dbf
skipping datafile 10; already restored to file /srv/nffs/oradata/ant12/data/NLWLDELFTFEWSModDat01_01.dbf
skipping datafile 11; already restored to file /srv/nffs/oradata/ant12/data/utility01_01.dbf
skipping datafile 12; already restored to file /srv/nffs/oradata/ant12/data/xdb01.dbf
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00005 to /srv/nffs/oradata/ant12/data/tools01.dbf
channel ORA_DISK_1: reading from backup piece /media/oracle_backups/ant12/backupset_4.rman
channel ORA_DISK_1: piece handle=/media/oracle_backups/ant12/backupset_4.rman tag=TAG20120206T154606
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:07
Finished restore at 2012/02/06 16:16:00

RMAN> alter database open;
database opened

How easy was that then?

What happens if I’ve lost the SYSTEM tablespace then? Let’s see:

RMAN> shutdown

database closed
database dismounted
Oracle instance shut down

RMAN> host "mv /srv/nffs/oradata/ant12/data/system01.dbf /srv/nffs/oradata/ant12/data/system01.dbf.old";
host command complete

RMAN> startup

connected to target database (not started)
Oracle instance started
database mounted
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of startup command at 02/06/2012 16:20:45
ORA-01157: cannot identify/lock data file 1 - see DBWR trace file
ORA-01110: data file 1: '/srv/nffs/oradata/ant12/data/system01.dbf'

As before, we have “lost” a data file, this time making up the SYSTEM tablespace. Can we recover from this? Of course!

RMAN> restore database;

Starting restore at 2012/02/06 16:39:43
using channel ORA_DISK_1

skipping datafile 2; already restored to file /srv/nffs/oradata/ant12/data/sysaux01.dbf
...
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00001 to /srv/nffs/oradata/ant12/data/system01.dbf
channel ORA_DISK_1: reading from backup piece /media/oracle_backups/ant12/backupset_3.rman
channel ORA_DISK_1: piece handle=/media/oracle_backups/ant12/backupset_3.rman tag=TAG20120206T153851
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:02:36
Finished restore at 2012/02/06 16:42:20

And now I can open up the database again for use, or can I?

RMAN alter database open;

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of startup command at 02/06/2012 16:43:49
ORA-01113: file 1 needs media recovery
ORA-01110: data file 1: '/srv/nffs/oradata/ant12/data/system01.dbf'

It appears not! Even though I’ve recovered the SYSTEM tablespace file(s) I still have potential updates etc in the online redo logs. A quick recover database command should resolve any issues:

RMAN> recover database;

Starting recover at 2012/02/06 16:45:29
using channel ORA_DISK_1

starting media recovery
media recovery complete, elapsed time: 00:00:02

Finished recover at 2012/02/06 16:45:32

Now, I should be able to open up the database:

RMAN> alter database open;
database opened

Job done. All recovered and usable.

It is not possible to recover a single data file, or tablespace, from a cold backup if the database has been updated in any way since the cold backup was taken unless the database is running in archivelog mode.

If the database is in NOARCHIVELOG mode, then all you can do is restore the complete database. The next part gives a demonstration of this in action.

In ARCHIVELOG mode, RMAN will notice if any of the files have been updated, as shown in the following example where changes have been made to the database since the cold backup and “suddenly” a data file goes missing.

In this example, the user norman created a new table named test and added a single row to it. Then the file making up the users tablespace was renamed.

RMAN> startup;

connected to target database (not started)
Oracle instance started
database mounted
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of startup command at 02/06/2012 20:26:48
ORA-01157: cannot identify/lock data file 6 - see DBWR trace file
ORA-01110: data file 6: '/srv/nffs/oradata/ant12/data/users01.dbf'

As we are running in archivelog mode, we can recover only the users tablespace from the cold backup, and using the archive logs, recover the transactions.

RMAN> restore tablespace users;

Starting restore at 2012/02/06 20:28:40
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=18 device type=DISK
...
channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_DISK_1: restoring datafile 00006 to /srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: reading from backup piece /media/oracle_backups/ant12/backupset_4.rman
channel ORA_DISK_1: piece handle=/media/oracle_backups/ant12/backupset_4.rman tag=TAG20120206T154606
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:07
Finished restore at 2012/02/06 20:28:49

RMAN> recover tablespace users;

Starting recover at 2012/02/06 20:29:16
using channel ORA_DISK_1

starting media recovery
media recovery complete, elapsed time: 00:00:00

Finished recover at 2012/02/06 20:29:16

RMAN> alter database open;
database opened

That saves time as all we had to do was recover a single tablespace, which in this case consists of only one data file, and then reapply the committed transactions from the archivelogs which, handily, were still online. Had they not been online, RMAN would have found them in a backup and used those to recover the database.

This, of course, relies on a complete set of archived logs being found either online or in a backup. If any are missing, the database cannot be recovered fully.

Oracle RMAN for Beginners – Part 2

Norman — Mon, 06 Feb 2012 16:02:33 +0000

At the end of Oracle RMAN for Beginners – Part 1 I was ready to begin a backup of the ant12 database. Read on …

Cold Backup – Backupset Type

There are two different types of cold backup. The first uses RMAN’s own internal format for the dump files – a backupset – the other uses image copies of the individual database files. In this part I’ll concentrate on a backuset type of cold backup. I’ll look at image copies later.

A cold backup is always a full backup. To take a cold backup the database needs to be MOUNTed. This is slightly different from a normal cold backup using the OS tools to copy files, as the database should be SHUTDOWN in that case.

A cold backup is all that you can do when the database is running in NOARCHIVELOG mode, however, you can take a cold backup of a database running in ARCHIVELOG mode using the commands shown below.

RMAN> connect target /
connected to target database: ANT12 (DBID=2799264292)

RMAN> shutdown;
database dismounted
Oracle instance shut down

RMAN> startup mount;
connected to target database (not started)
Oracle instance started
database mounted
...

The following command creates a new backupset in the FRA. The files will be written to the FRA in a directory named backupset/SID.

RMAN> backup full database;

Starting backup at 2012/02/06 09:58:40
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=18 device type=DISK
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00010 name=/srv/nffs/oradata/ant12/data/NLWLDELFTFEWSModDat01_01.dbf
...
input datafile file number=00006 name=/srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: starting piece 1 at 2012/02/06 09:58:41
channel ORA_DISK_1: finished piece 1 at 2012/02/06 09:59:16
piece handle=/srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_06/o1_mf_nnndf_TAG20120206T095841_7lz92krk_.bkp tag=TAG20120206T095841 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:35
Finished backup at 2012/02/06 09:59:16

Starting Control File and SPFILE Autobackup at 2012/02/06 09:59:16
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_06/o1_mf_s_774524380_7lz93qbg_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/06 09:59:23

And that’s all there is to it. The backup file(s) will be created in the FRA as detailed above by various piece handle messages. You can see the backup details using the list backup command:

RMAN> list backup;

List of Backup Sets
===================

BS Key  Type LV Size       Device Type Elapsed Time Completion Time    
------- ---- -- ---------- ----------- ------------ -------------------
6       Full    365.34M    DISK        00:00:35     2012/02/06 09:59:16
        BP Key: 6   Status: AVAILABLE  Compressed: NO  Tag: TAG20120206T095841
        Piece Name: /srv/nffs/flashback_area/ant12/ANT12/backupset/2012_02_06/o1_mf_nnndf_TAG20120206T095841_7lz92krk_.bkp
  List of Datafiles in backup set 6
  File LV Type Ckp SCN    Ckp Time            Name
  ---- -- ---- ---------- ------------------- ----
  1       Full 847342     2012/02/06 09:39:40 /srv/nffs/oradata/ant12/data/system01.dbf
...
  12      Full 847342     2012/02/06 09:39:40 /srv/nffs/oradata/ant12/data/xdb01.dbf

BS Key  Type LV Size       Device Type Elapsed Time Completion Time    
------- ---- -- ---------- ----------- ------------ -------------------
7       Full    9.70M      DISK        00:00:03     2012/02/06 09:59:20
        BP Key: 7   Status: AVAILABLE  Compressed: NO  Tag: TAG20120206T095916
        Piece Name: /srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_06/o1_mf_s_774524380_7lz93qbg_.bkp
  SPFILE Included: Modification time: 2012/02/06 09:40:06
  SPFILE db_unique_name: ANT12
  Control File Included: Ckp SCN: 847342       Ckp time: 2012/02/06 09:39:40

The backup will be created in DB_RECOVERY_DEST/SID/backupset. A separate backup took place to copy the controlfile and spfile so that details of this current backup remain safe.

Note in most cases the SID will be used to determine the output directory in the FRA as specified by the database parameter DB_RECOVERY_DEST. However it is not ORACLE_SID that is actually used but the DB_UNIQUE_NAME initialisation parameter. I used SID as it’s easier to type!

It is possible to avoid backing up to the FRA by specifying a FORMAT parameter:

RMAN> backup format='/media/oracle_backups/ant12/ant12_backupset.rman' full database;

Starting backup at 2012/02/06 12:30:20
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
input datafile file number=00010 name=/srv/nffs/oradata/ant12/data/NLWLDELFTFEWSModDat01_01.dbf
...
input datafile file number=00006 name=/srv/nffs/oradata/ant12/data/users01.dbf
channel ORA_DISK_1: starting piece 1 at 2012/02/06 12:30:20
channel ORA_DISK_1: finished piece 1 at 2012/02/06 12:31:45
piece handle=/media/oracle_backups/ant12/ant12_backupset.rman tag=TAG20120206T123020 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:01:25
Finished backup at 2012/02/06 12:31:45

Starting Control File and SPFILE Autobackup at 2012/02/06 12:31:46
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_06/o1_mf_s_774524380_7lzl1lcg_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/06 12:31:47

Unfortunately, if you look carefully at the last few lines of output, the controlfile and spfile
autobackup have still gone to the FRA and not to the desired location. What to do?

You can temporarily turn off the controlfile autobackup, run a database backup followed by a manual controlfile (and spfile) backup and then turn controlfile autobackup back on, as follows. You don’t have to use a run {} block as RMAN will happily accept all the commands separately. (This is not always the case!)

RMAN> run {
2> configure controlfile autobackup off;
3> backup format = '/media/oracle_backups/ant12/backupset_4.rman' full database;
4> backup format = '/media/oracle_backups/ant12/cf_backup_manual.f' current controlfile;
5> configure controlfile autobackup on;
}
 
old RMAN configuration parameters:
CONFIGURE CONTROLFILE AUTOBACKUP ON;
new RMAN configuration parameters:
CONFIGURE CONTROLFILE AUTOBACKUP OFF;
new RMAN configuration parameters are successfully stored

Starting backup at 2012/02/06 15:46:06
using channel ORA_DISK_1
...
piece handle=/media/oracle_backups/ant12/backupset_4.rman tag=TAG20120206T154606 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:25
Finished backup at 2012/02/06 15:46:32

Starting backup at 2012/02/06 15:46:32
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
including current control file in backup set
channel ORA_DISK_1: starting piece 1 at 2012/02/06 15:46:33
channel ORA_DISK_1: finished piece 1 at 2012/02/06 15:46:40
piece handle=/media/oracle_backups/ant12/cf_backup_manual.f tag=TAG20120206T154632 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:07
Finished backup at 2012/02/06 15:46:40

old RMAN configuration parameters:
CONFIGURE CONTROLFILE AUTOBACKUP OFF;
new RMAN configuration parameters:
CONFIGURE CONTROLFILE AUTOBACKUP ON;
new RMAN configuration parameters are successfully stored

There has to be an easier way to do this doesn’t there? What about when you don’t know for sure that the configuration parameter is on or off? I’m sure there’s a way, and if there is, I’ll find it.

UPDATE there is a way. Simply use the set command instead of configure. For example, the above should be replaced by:

RMAN> run {
2> set controlfile autobackup off;
3> backup format = '/media/oracle_backups/ant12/backupset_4.rman' full database;
4> backup format = '/media/oracle_backups/ant12/cf_backup_manual.f' current controlfile;
}

The set remains in force until the end of the session, or, as in this case, until the end of the run block.

Anyway, you now have a cold backup of the database and a safety copy of the controlfile. Unfortunately, perhaps, the backup of the controlfile didn’t take a backup of the spfile. To do that you need to add in a separate manual copy of the spfile similar to the following:

RMAN> backup format = '/media/oracle_backups/ant12/sp_backup_manual.f' spfile;

Starting backup at 2012/02/06 15:58:31
using channel ORA_DISK_1
channel ORA_DISK_1: starting full datafile backup set
channel ORA_DISK_1: specifying datafile(s) in backup set
including current SPFILE in backup set
channel ORA_DISK_1: starting piece 1 at 2012/02/06 15:58:31
channel ORA_DISK_1: finished piece 1 at 2012/02/06 15:58:32
piece handle=/media/oracle_backups/ant12/sp_backup_manual.f tag=TAG20120206T155831 comment=NONE
channel ORA_DISK_1: backup set complete, elapsed time: 00:00:01
Finished backup at 2012/02/06 15:58:32

Starting Control File and SPFILE Autobackup at 2012/02/06 15:58:33
piece handle=/srv/nffs/flashback_area/ant12/ANT12/autobackup/2012_02_06/o1_mf_s_774524380_7lzy59go_.bkp comment=NONE
Finished Control File and SPFILE Autobackup at 2012/02/06 15:58:34

Did you notice? The backup of the spfile includes a backup of the controlfile. Is nothing consistent with RMAN?

Coming soon, restoring and recovering a database from a cold backup.

Oracle RMAN for Beginners – Part 1

Norman — Mon, 06 Feb 2012 15:09:58 +0000

Introduction

One of the things a DBA needs to be aware of, is RMAN. This has been around since Oracle 8 (or was it 8i?) and has been improving since then. It’s almost pretty good at 11.2!

One of the things that I, as a DBA, need to get to grips with is RMAN. Most of the work I’ve been doing for the last few years have not involved very much in the way of RMAN usage. It’s time I put an end to my almost complete ignorance.

I have set myself up a small test system where I can do my own RMAN training. I’ll be writing up my results and findings as I go along. Feel free to correct me where I get things wrong – which I no doubt will!

Sensible Prerequisites

I have found a few sensible prerequisites to using RMAN. For best results, set your environment up in a similar manner.

I’ll be using a database named ant12 running on a server named hubble.

The first irritation I have with RMAN is it’s use of dates. They are never expanded to how I like them to be. So run the following command in a shell session:

hubble> export NLS_DATE_FORMAT='yyyy/mm/dd hh24:mi:ss'

Next up, we have the problem that on some Linux/Unix systems the rman command is actually in two places, one in ORACLE_HOME, the other under the X11 installation. Usually, X11 is higher up the PATH than ORACLE_HOME, so we need to be sure we are using the right one.

hubble> alias rman='$ORACLE_HOME/bin/rman'

Using single quotes in the command allows the evaluation of ORACLE_HOME to be carried out at run time – when the rman command is called – not at define time. That way, whatever ORACLE_HOME is current at the time of running RMAN will be the correct one for the database.

Now we need to make sure that we have set the Oracle environment for the database to be backed up.

# I'll be using a database called ant12.
hubble> . oraenv
ORACLE_SID = [ant12] ? ant12
The Oracle base remains unchanged with value /srv/oracle

Oracle advise that the database should be using a FLASH RECOVERY AREA (FRA), so let’s set the database up to use one:

hubble> sqlplus / as sysdba

SQL> -- Commands must be typed in the following order ....
SQL> alter system set log_archive_duplex_dest='';
System altered.

SQL> alter system set log_archive_dest='';
System altered.

SQL> alter system set db_recovery_size=19G;
System altered.

SQL> alter system 
set db_recovery_dest='/srv/nffs/flashback_area/ant12';
System altered.

There is a reason for not having the archived logs stored within the FRA, it becomes a single point of failure. However, there are a few reasons for doing so, the main one being that you have all files necessary for a recovery in the same place.

If you are using the FRA for archived logs, it is probably a good idea to be mirroring the FRA.

For my test system, I am putting the archived logs into the FRA.

SQL> alter system set
log_archive_dest_1='LOCATION=USE_DB_RECOVERY_FILE_DEST';
System altered.

SQL> alter system set log_archive_dest_state_1=enable;
System altered.

Check and Amend Configuration

RMAN comes configured with reasonably sensible defaults. However, they are not as desired for my use, so the first thing to do is look at the default settings, and change them.

# Run the correct version of RMAN for the database
#the display the current default settings:
hubble> rman target /

RMAN> show all;

RMAN configuration parameters for database with db_unique_name ANT12 are:
CONFIGURE RETENTION POLICY TO REDUNDANCY 1; # default
CONFIGURE BACKUP OPTIMIZATION OFF; # default
CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default
CONFIGURE CONTROLFILE AUTOBACKUP OFF; # default
CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F'; # default
CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
CONFIGURE MAXSETSIZE TO UNLIMITED; # default
CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
CONFIGURE COMPRESSION ALGORITHM 'BASIC' AS OF RELEASE 'DEFAULT' OPTIMIZE FOR LOAD TRUE ; # default
CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/srv/oracle/product/11gR1/db/dbs/snapcf_ant12.f'; # default

We are advised, by Oracle, that we should configure RMAN to always backup the control file (which includes the spfile – if one is in use) if we are not using an RMAN catalogue. This is because without a catalogue we store details of backups in the control file, so we don’t want to lose those.

We will also configure RMAN to prevent the copying of virgin blocks. These are blocks in data files which have never been used at all. (Being used and then emptied means a block is no longer virgin.) This can help to reduce backup times.

We will do all of them in one go, using a run {} block.

RMAN> run {
2> configure controlfile autobackup on;
3> configure backup optimization on;
4> }

new RMAN configuration parameters:
CONFIGURE CONTROLFILE AUTOBACKUP ON;
new RMAN configuration parameters are successfully stored

new RMAN configuration parameters:
CONFIGURE BACKUP OPTIMIZATION ON;
new RMAN configuration parameters are successfully stored

We are ready to backup the ant12 database to the FRA.

Docbook – Creating Indexes

Norman — Fri, 13 Jan 2012 20:10:24 +0000

I’ve been doing a bit of Docbook work recently – converting a paper based manual into a Docbook one that can then be used to generate all kinds of different output from the same input file. Very useful.

I needed to create an index that correctly reflected the contents of the new format rather than simply copying the old one – where the pages would no longer have matched up.

I found a couple of good articles, namely:

http://www.xml.com/pub/a/2004/07/14/dbndx.html

and, of course, this one:

http://www.sagehill.net/docbookxsl/GenerateIndex.html

UPDATE: Of all the people you would expect to get it right, Bob Stayton is the one. Except he got it wrong. In the link above to www.sagehill.net, there is an example thus:

class="startofrange" id="makestuff">
 Makefiles

 ... 
class="endofrange" startref="makestuff">
 Makefiles

It should be as follows without the on the end of page range indexterm. If you do it with a primary, you get the page range as expected, but with the final page duplicated, as in:

Subject matter 123-127, 127

Remove the primary and it just works.

Raspberry Pi Goes Into Manufacturing – Yippee!

Norman — Wed, 11 Jan 2012 07:38:29 +0000

It’s finally coming! Raspberry Pi has now officially started manufacture of the first production boards. See http://www.raspberrypi.org/archives/509 for details.

Sadly, you can also see there why the tax situation in the UK (or possibly because of the EU) is preventing the charity from manufacturing these boards in the UK.

And, equally sadly, it may also be true that certain UK manufacturers don’t seem to want the extra work either.

How sad are we in the UK?

Cheers.

Why Do I Have To Suffer Your Children?

Norman — Sun, 08 Jan 2012 15:19:53 +0000

I’m trapped on a plane for four and a bit hours and your bloody child has been screaming it’s head off for most of the way.

There’s little chance of me being able to sleep at all on this flight but somehow that’s ok with you because “she’s only 18 months old.”

Do you think that I, and the other passengers around you, think that that is an acceptable reason for suffering 4 and a bit hours of continual screaming? The answer, if you give a toss, is a resounding “NO IT FUCKING ISN’T!”

If I make a noise around your bloody child, you will tell me to “be quiet or I will wake the baby”, so what makes you think that it’s perfectly ok the other way around?

Don’t even think of telling me that she’s too young to know better. If that’s the case, she’s too bloody young to be keeping every passenger aboard this Boeing 737-800 awake – put the damned child in the bloody kennels next time you want a holiday you selfish and inconsiderate pair of idiots!

(Can you tell I’m seriously pissed off?)

Cheers.

PS. It’s 22:05 on new year’s eve 2011 and I need to get some sleep before the drive home from Manchester airport and your fucking child is getting right on my tits. Shut it the fuck up NOW!

PPS. This post written on my Galaxy Tab while airbourne “somewhere over Europe”. Obviously I can’t post it from a plane, so you’ll get to see it some time after we have landed.

Slightly Weird Oracle Stuff

Norman — Wed, 14 Dec 2011 09:40:11 +0000

I knew you could do this:

SQL> select 1234567890 as abc from dual;

       ABC
----------
1234567890

SQL> select 1234567890 abc from dual;

       ABC
----------
1234567890

But I didn’t know that this worked as well:

SQL> select 1234567890abc from dual;

       ABC
----------
1234567890

So I did a bit of playing and discovered that there is a difference if the alias is D or F but no other single character:

SQL> select 1234567890d, 1234567890f, 1234567890p from dual;

1234567890D 1234567890F          P
----------- ----------- ----------
 1.235E+009  1.235E+009 1234567890

This shows the values in Scientific notation when D or F is used as an alias in this manner, but not if used in this manner:

SQL> select 1234567890 d, 1234567890 f, 1234567890 p from dual

         D          F          P
---------- ---------- ----------
1234567890 1234567890 1234567890

Then it gets stranger:

SQL> select 1234567890df from dual;

         F
----------
1.235E+009


SQL> select 1234567890fd from dual

         D
----------
1.235E+009


SQL> select 1234567890fa from dual

         A
----------
1.235E+009

I get the impression that a trailing F or D on a number means “display as floating point or decimal” then the F/D is dropped and the A used as a label. I can’t find this in the docs though.

Works with strings as well but the F/D thing doesn’t appear with strings. Doesn’t work – for obvious reasons – with column names.

UPDATE: Thanks to Maxim on the Oracle-L list, the answer is here

UPDATE 2: Thanks to Jonathan Lewis also on the Oracle-L list, it seems that Tanel Poder has also come across this. On his blog here.

Cheers,

The Green Thing

Norman — Tue, 13 Dec 2011 15:58:30 +0000

This is doing the rounds on email at the moment, but it made me laugh. Grumpy old sod that I am!

Checking out at the store, the young cashier suggested to the older woman that she should bring her own grocery bags because plastic bags weren’t good for the environment. The woman apologised and explained, “We didn’t have this green thing back in my earlier days.”

The clerk responded, “That’s our problem today. Your generation did not care enough to save our environment for future generations.” She was right — our generation didn’t have the green thing in its day.

Back then, we returned milk bottles, soda bottles and beer bottles to the store. The store sent them back to the plant to be washed and sterilised and refilled, so it could use the same bottles over and over. So they really were recycled. But we didn’t have the green thing back in our day.

We walked up stairs, because we didn’t have an escalator in every store and office building. We walked to the grocery store and didn’t climb into a 300-horsepower machine every time we had to go two blocks. But she was right. We didn’t have the green thing in our day.

Back then, we washed the baby’s nappies [diapers] because we didn’t have the throw-away kind. We dried clothes on a line, not in an energy gobbling machine burning up 3000 Watts every hour — wind and solar power really did dry our clothes back in our early days. Kids got hand-me-down clothes from their brothers or sisters, not always brand-new clothing. But that young lady is right. We didn’t have the green thing back in our day.

Back then, we had one TV, or radio, in the house — not a TV in every room. And the TV had a small screen the size of a handkerchief (remember them?), not a screen the size of the state of Montana^*** . In the kitchen, we blended and stirred by hand because we didn’t have electric machines to do everything for us. When we packaged a fragile item to send in the mail, we used wadded up old newspapers to cushion it, not Polystyrene [Styrofoam] or plastic bubble wrap. Back then, we didn’t fire up an engine and burn gasoline just to cut the lawn. We used a push mower that ran on human power. We exercised by working so we didn’t need to go to a health club to run on treadmills that operate on electricity. But she’s right. We didn’t have the green thing back then.

We drank from a fountain when we were thirsty instead of using a cup or a plastic bottle every time we had a drink of water. We refilled writing pens with ink instead of buying a new pen, and we replaced the razor blades in a razor instead of throwing away the whole razor just because the blade got dull. But we didn’t have the green thing back then.

Back then, people took the tram [streetcar] or a bus, and kids rode their bikes to school or walked instead of turning their parents into a 24-hour taxi service. We had one electrical outlet in a room, not an entire bank of sockets to power a dozen appliances. And we didn’t need a computerised gadget to receive a signal beamed from satellites 2,000 miles out in space in order to find the nearest pizza joint.

But isn’t it sad the current generation laments how wasteful we old folks were just because we didn’t have the green thing back then?

Please forward this on to another selfish old person who needs a lesson in conservation from a smart-ass young person.

Remember: Don’t make old people mad. We don’t like being old in the first place, so it doesn’t take much to make us cross.

*** Yes, I know, TVs today use far less power than the old CRT versions of yesteryear – but this is humour ok? And why let facts get in the way>

Cheers.

Adobe Air Applications Can’t Find Passwords etc?

Norman — Thu, 08 Dec 2011 15:56:42 +0000

TweetDeck for Linux, for example. Refused to start on me recently because it couldn’t find or access the location where I had saved my encrypted details. The problem is caused by Adobe Air not being able to find a running daemon for Gnome Keyring or KWallet, or a corrupted wallet database.

Full details here http://kb2.adobe.com/cps/492/cpsid_49267.html – works for me!

In my case, simply restarting KWalletManager worked fine. And restarting TweetDeck of course – it’s unable to pick up the fact that the wallet is available.

Cheers,
Norm.

Deprecated Parameter Warning on Database Startup

Norman — Mon, 21 Nov 2011 10:34:49 +0000

You know the feeling, your 10g or 11g database displays a warning message about your use of deprecated parameters at startup, but it doesn’t say which parameters are deprecated?

You could look through the manuals to find the list of all deprecated parameters then go hunting in your spfile/pfile for those and remove them, or, you could simply look in the alert.log.

...
Starting up ORACLE RDBMS Version: 10.2.0.4.
...
Deprecated system parameters with specified values:
  log_archive_start
End of deprecated system parameter listing
PMON started with pid=2, OS id=11541
...

There are a few more at 11g:

...
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 ...
...
Deprecated system parameters with specified values:
  log_archive_start
  hash_join_enabled
  max_enabled_roles
  background_dump_dest
  user_dump_dest
End of deprecated system parameter listing
PMON started with pid=2, OS id=5292
...

These databases were converted from 9.2.0.8 to 10g and 11g respectively. So, now you know what the deprecated parameters are, go fix them!

Cheers.

Does Your Volume Control Freeze on KDE?

Norman — Wed, 16 Nov 2011 14:34:54 +0000

I’ve been using KDE4 on OpenSuse (11.4 currently) for some time and never had any problems. After a recent patch fix to the system, the volume control buttons on my Dell Vostro stopped responding. Normally pressing volume up or volume down worked instantaneously. Stop and go still worked fine, but not volume.

What would happen is that the on screen indicator would appear after about 45 seconds or so – but the volume wouldn’t change until it did, then, it would go to maximum or minimum – depending on which button I’d been hammering for 45 seconds!

A similar thing would happen if I tried to use the Volume control applet in the task bar. Hmmm.

The solution is fairly simple:

Stop the kmix process by right-clicking it in the task bar, and choosing the quit option.
cd ~/.kde4/share/apps rm -R kmix*
Restart Kmix by pressing ALT-F2 and typing kmix into the dialogue that appears. After a few seconds, you get your volume control icon back in the task bar and everything works fine.

The problem appears to have been caused by a configuration change which left problems when old config was found in the assorted config files – but don’t quote me on that!

Cheers.

Indoor Beekeeping? It Won’t Catch on!

Norman — Thu, 10 Nov 2011 18:12:39 +0000

Check this link for details. I’m saying nothing!

Except, maybe, I really can’t see it taking off.

Well, ok then. How do you stop the queen laying in the honey comb? There doesn’t appear to be a queen excluder – so your honey will have all sorts of stuff in it! Larvae, eggs, etc. It really won’t work!

Cheers.

NetBeans Complains That it Can’t Find Java.Lang in your Android Project.

Norman — Fri, 04 Nov 2011 15:55:17 +0000

Using NetBeans (version 7.0.1) to develop Android applications and for no apparent reason, a project that worked yesterday won’t run any more. It compiles without error but refuses to run – whether or not the emulator is running – and doesn’t produce an error. Hmmm. In the source code for the main Java class (yes, I said Java – anyone who knows me will be having a fit right now!) the very first line is flagged with a red (ok, pink!) dot. The line itself simply says:

package uk.co.dunbarit.myprojectname;

Rolling the mouse over the line pops up a helpful hint which states that the system cannot find java.lang in classpath or bootclasspath.

Now, as my work colleagues will tell you, I’m not a Java developer by any means. But this was working yesterday, so why not today?

Right-click the project name in the project tab, and select the option – near the bottom – to rebuild broken project. This pops up with a dialogue box upon which appears the message that I should “update the project using SDK tools“. Easy? What does that mean?

What it means is this. I have to:

Close the project in NetBeans and open a terminal session.
Navigate to the Android SDK’s tools directory. In there you’ll find the android utility.
./android update project --name MyProjectName --path full_path_to_project

Job done! Reopen the project in NetBeans and run it. Yippee! By the way, the --path parameter is set to the location where the AndroidManifest.xml file can be found.

It appears that the Android SDK updated itself thus rendering something that was legal as no longer legal. Nice one – thanks. I’m having enough trouble with Java without having silent updated bollox me as well!

Cheers.

Can We Have 10g and 11g on the Same Server?

Norman — Wed, 02 Nov 2011 15:42:04 +0000

If you install 10g and 11g on the same server, which one do you wish to supply the executables for “oraenv”?

If you install 10g first and allow 11g to overwrite the files in /usr/local/bin when you run “root.sh” then when you eventually call oraenv to set a 10g environment, you get a warning that “$ORACLE_HOME/bin/orabase” cannot be found.

If you allow the 10g files to overwrite the 11g ones, you don’t set ORACLE_BASE. I wonder what problems that might cause?

Cheers.

Removing “Rogue” Expensive Oracle Options After Upgrading to 10.2.0.x.

Norman — Wed, 02 Nov 2011 15:37:26 +0000

Install 10.2.0.1 – the base release – of Oracle and deselect the various Enterprise Edition Options – these cost money and we don’t like that, especially if we don’t use them.

Patching to 10.2.0.5 (in my installation – it could be different with previous versions) silently adds OLAP, Data Mining etc back into the mix. Not good – especially if/when Oracle decide to audit your licenses. You pay for what you install whether used or not.

So, to remedy the problem, do this:

cd $ORACLE_HOME/rdbms/lib make -f ins_rdbms.mk olap_off rat_off rac_off dm_off sdo_off ioracle

You may see an error telling you that SDO can’t be turned off. That’s fine, at least we tried! The various options are:

olap_off = OLAP
rat_off = Real Application Testing
rac_off = Real Application Cluster
dm_off = Data Mining
sdo_off = Spatial

There are others in the make file, but these are the ones I’m mostly interested in not having!

Cheers.

Norm’s Law of Computing

Norman — Thu, 27 Oct 2011 16:37:24 +0000

Software efficiency halves every 18 months thus compensating for Moore’s Law.

Cheers.

A Day in London with Cary Millsap

Norman — Fri, 09 Sep 2011 14:41:34 +0000

Yesterday I had the pleasure of a day trip to London to attend the Mastering Oracle Trace Data session with Cary Millsap. The day got off to a great start as I arrived at the railway station in Leeds very early to find my train waiting for me at the platform. Excellent! I got on and 20 minutes before departure time, we headed on out of Leeds for London. WTF? The announcement that this was the 06:40 to London Kings Cross and not the 07:00 cleared things up a little.

Guess what? My ticket was only valid on the 07:00. However, I sought out the guard and she checked my ticket, her timetable and said that “as it’s only a difference of 6 minutes, don’t worry about it”. Result! Nice lady.

After arriving in London, finding the location for the session wasn’t too bad, well if you exclude the £4 per trip to do two stations on the Tube, which was absolutely jam packed. I had to let one train go past as there was simply no room on it for anyone else. Luckily the next one wasn’t as packed. (Well, just!)

The session itself was an exceedingly interesting day. Cary is an excellent speaker and made what could have been a pretty dry day, very interesting indeed. There were 16 of us in total including one Doug Burns – who I would have thought already knew all this stuff!

The lunchtime chicken and mash was just that, chicken and mash. With a spot of gravy and a slice of lemon thrown in for good measure. I decided to get my jacket and nip out for something else, and a little walk. The room was locked.

In the afternoon session we looked at lots of ways to use the tools that we would be taking home with us. These are the MR Tools and MR Trace available from www.method-r.com – they appear exceedingly useful indeed. I’m currently waiting to obtain my license so that I can get hold of them and start trying them out on a few trace files from my past.

Mr Trace won’t be much use to me though, it’s an add on for SQL Developer which I don’t use, being a Toad man myself. However, I have passed Method R’s details on to Quest’s product manager for a potential inclusion in a future version of Toad, so maybe one day!

I really think the course should have been over a couple of days rather than just one, as all too soon, we were all departing for home. But not before we had picked up our exclusive T shirts (not available in the shops!) and posed for a group photo.

Cheers.

2011 Honey Harvest – Let it Begin!

Norman — Thu, 25 Aug 2011 12:57:13 +0000

On Sunday past, I took 11 full & capped super frames of lovely honey from the original hive, and checked the new hive. They are a bit behind and have not yet capped anything off. Hmmm, what are they up to I wonder?

The original hive got another full set of super frames to draw out and fill up. Seeing as how it’s getting late on in the year, I’m not sure we’ll get much more, if anything off of them – but you never know.

The third hive (well, nucleus box) is going to Peter as soon as we can get the little b*ggers to stay inside so that I can block off the entrance and get them transported. They are up early every morning and they stay out much later that I do at night! Women!

Jonathan Lewis Has A New Book Coming Soon ….

Norman — Wed, 20 Jul 2011 09:26:18 +0000

Great news! (At least for us Oracle types anyway.)

Jonathan Lewis has a new book coming out. The working title appears to be “A look at the internal mechanics of the important bits of Oracle for people who aren’t planning to become rocket scientists but who do want to do a little more than just push buttons in OEM” but this will probably change by November when it is due to be published, all going well.

Just in time for Christmas!

Check out the details here.

Cheers.

This Page Intentionally Left Blank …

Norman — Wed, 29 Jun 2011 10:38:12 +0000

Except, it’s not!

WTF is it all about? You open a report, or a book and see a page with no text on it except this page intentionally left blank. Except, because of that bit of text, it’s no longer blank is it?

So why do they bother? Does the author/publisher think we readers are too stupid to realise that a page is blank without being told? Is it because they think we might assume that somehow, the printing process has mysteriously failed for that one page in particular and we may be missing some highly important information? Or what?

If they really must tell us that the page was never intended to have anything useful on it, surely this page has no useful information on it could be a better idea. Except, pedants like me, might assume that because of what it is telling us, that it does indeed contain useful information.

I give up. Just leave the blank pages as blanks and we’ll probably get the message that it has no content and is simply being used to ensure that the first page of the next section, part, chapter, whatever, starts on the generally accepted face of the page. It’s not rocket science!

Cheers.

Wondering Why The Oracle Databases Won’t Start With A SLES Reboot?

Norman — Fri, 24 Jun 2011 14:27:10 +0000

Me too. Took ages to hit the “duh” moment, then it became pretty obvious! The file /etc/init.d/oracle also known as rcoracle to root users can be used to do a number of things such as starting the databases, starting the (default LISTENER) listener, CRS etc but, as I eventually found out, you have to configure it to do so!

The configuration file is /etc/sysconfig/oracle.

Most of the options are defaulted to off, except for the setting of kernel parameters (SET_ORACLE_KERNEL_PARAMETERS="yes") which is very useful.

Cheers.

How To Extract Details From /etc/oratab on Linux

Norman — Fri, 24 Jun 2011 14:22:17 +0000

Ever wanted to parse /etc/oratab but ignore all the comments and blank lines? So did I. Here’s how …

I can’t claim all the credit for this, it is based on something I was doing plus a bit of “stolen” code from SLES.

OLDIFS=$IFS
IFS=:
grep -v '^\(#\|$\)' /etc/oratab        |\
while read ORASID ORAHOME AUTOSTART
do
        ## Do what you like here with
        ##  $ORASID, $ORAHOME and $AUTOSTART ##
done
IFS=$OLDIFS

Cheers.

Does Your Windows Oracle Database Stay Down After A Server Reboot?

Norman — Tue, 21 Jun 2011 09:41:44 +0000

After a server reboot, your (windows) database stays down, even though you have it set to come up automatically – why? The services have started, just not the database.

The following is a clue:
sqlplus "/ as sysdba"

You now have two options to check, and both must be confirmed:

Is the oracle user a member of the ora_dba group?
Is sqlnet.ora correctly configured?

To check the first, proceed as follows (on the database server!):

Start->Programs->Administrative Tools->Computer Management.
Double-click Local Users and Groups.
Double-click Groups.
Double-click ora_dba (on the right) if it exists. If it doesn’t, it should – so have it added.
The list of members of the ora_dba group should appear. Make sure that the oracle user is listed. If not, add it to the group.
If you made changes, logout of the oracle user and back in again, then continue.

To correctly configure sqlnet.ora, make sure that the following line appears:
SQLNET.AUTHENTICATION_SERVICES=(NTS)

If it is not found, then the default is (NONE) which requires a username and password always be supplied when connecting as sysdba or sysoper.

If it is set to anything else, other than (ALL), then it should be ok – although some of the other settings may not use (NTS) – you need to test on your system to be sure.

Now that you can connect "/ as sysdba", you can check the oradim.log (%oracle_home%\database\oradim.log) to see if there are any of these errors corresponding to your server reboot times:

oradim.exe -startup ....... ORA-01017: invalid username/password; login denied

You may also find entries in the application pages of the event viewer with similar messages:
ORACLE. "CONNECT" DATABASE USER: "/" PRIVILEGE: NONE CLIENT_USER: oracle CLIENT_TERMINAL: STATUS: 1031

The alert log shows no errors whatsoever. It won’t, you didn’t get that far!

If the oradim.log shows no errors of the above kind, you may not have the database configured to autostart. So, delve into the registry. If the two following keys exist, and have the value true, then all should be well on the next restart. Otherwise add/edit them manually. Setting both to true ensures that when you start and stop the services the database will also start and stop.

When the server is rebooted, the services are told to stop, so they will stop the database cleanly before the server goes down. The reverse is true on a server startup, the services are told to start and they tell the database to come on up!

HKLM/software/oracle/key_ora_home_name/ora_oracle_sid_autostart HKLM/software/oracle/key_ora_home_name/ora_oracle_sid_shutdown

If all of the above pan out, next time you restart the server, the database will come up!

You can test without bouncing the server simply by shutting down and restarting the database service from control panel. On a restart of the service, the database should also be back up and running. If not, get into the oradim.log again, and fix the problem you find.

Cheers.

Does Your Hard Disc Lights Stay On For Ages After Booting Linux?

Norman — Mon, 20 Jun 2011 11:15:27 +0000

Mine did, not any more though. Read on.

I was just a tad annoyed at the sudden increase in s-l-o-o-o-o-w responses after starting OpenSuse 11.4 up. I noticed that the hard drive indicator LED was continually lit for about 5-10 minutes after bootup. Performance was abysmal during this time.

Top showed almost no CPU being used, so I looks ad vmstat instead. This was the result, on an idle (and slow) system:

vmstat -d disk- - -----IO------ cur sec sda8 0 898

Note: In the above, I’ve removed the individual reads and writes columns, to sort out the wheat from the chaff. I’m interested in I/Os overall. I’ve also removed the partitions that were showing 0 or 1 as the total I/O count.

The figures showed me that partition sda8 was getting hit to the tune of almost 900 I/O operations per second. Who or what is partition sda8?

df -h | grep -i sda8 /dev/sda8 251G 131G 108G 55% /data

Now I know, so what’s open on this mount point?

lsof /data COMMAND PID USER NODE NAME preload 385 root /data/VirtualBox/ScientificLinuxEnterprise6.vdi preload 385 root /data/VirtualBox/LinuxMint64bit.vdi

And having noted the above, why on earth was anything loading my virtual drives when I’m not even running VirtualBox at the moment?

ps -ef|grep -i 385 root 385 1 1 11:15 ? 00:00:11 /sbin/preload /var/cache/preload/prepared

A quick check of the internet showed that preload is a process to save boot time (how ironic) and that it can easily be disabled. So I disabled it. No more long boots now! To disable preload, run yast and select System Services (Runlevel).

Tick the Expert Mode radio button.
Scroll down to find boot.startpreload
At the bottom of the screen, uncheck the B box.
Click OK.

Cheers.

Getting Amazon “Kindle For PC” running on Linux.

Norman — Thu, 19 May 2011 09:08:40 +0000

How hard can it be? Well, as it turns out, not very. The steps are as follows:

Install, or check, that you have wine version 1.3 installed. The command wine --version gives details, assuming that you already have it installed.
Download “Kindle for PC” from Amazon’s web site.
Double-click the Kindle exe file to install it under wine. If you see errors about “winesetup encountered an error” just ignore them.

You will have a new entry in your applications menu for Kindle. How difficult was that? Here is Kindle for PC running on OpenSuse 11.4 with a KDE desktop:

Kindle for PC, on Linux. Click for full sized image.

A couple of things don’t work as they do in Windows, for example, shopping in the Kindle Store opens up in your default browser, which is not really a problem – you get full screen browsing. As far as I can see, synchronising with your other Kindle(s) works fine – I’m running Kindle for Android as well as Kindle for PC and they happily sync with each other.

I also see the following wine problem whenever I start Kindle, but this occurs on a couple of other wine programs as well, and did previously in OpenSuse 11.2 after a patch to wine. So far, nothing has been affected.

The error that occurs in wine when applications are executed.

Cheers.

Bees Coming Out Of Our Ears!

Norman — Thu, 19 May 2011 08:47:35 +0000

One of Alison’s ex work colleagues, Andy, keeps bees. So far this year he has had 4 swarms! We took advantage of his generosity and restocked our recently dead hive with one of his swarms, and then, a couple of days later we obtained another of his swarms in a (borrowed – thanks Peter) nucleus box. We now have two and a half hives.

Our own hive, looks to have requeened itself – my last inspection showed no red queen, and what appeared to be an unmarked queen laying. Hmmm. There were, however, no signs of any queen related waxworks going on. I’m confused!

Cheers.

Do We Have A New Queen, Or What?

Norman — Sun, 01 May 2011 16:13:48 +0000

I’ve been checking the girls today – the weather is lovely and warm and they are flying around all over the place, returning with lots of pollen and, hopefully, nectar.

We’ve been away for a couple of weeks and before we went, I fed them on about 5 pounds of honey and put a super on with last years frames – fresh (!) from extraction – in case the girls needed something to do while we were away. Unfortunately, due to being in a hurry and getting ever so slightly confused, I put the queen excluder above the super rather than below it. Sigh! Today, I rebuilt the hive correctly after checking that all was well in camp.

It seems that the queen has not moved up into the super anyway as there was no sign of laying in the super frames. This is good news, there is, however, a little honey in a couple of frames, so they seem to be getting on with things already!

Now we have (or had?) a red marked queen in our original swarm and this makes her 4 years old this year which, I’m told, is past her best. However, I didn’t see her at all today when I was inspecting the hive but we do have larvae in various stages of development from tiny little newly hatched ones up to lots of fully developed and capped over cells – so someone is laying. There are a few drones too, but not too many. So far, there is very little drone comb around. So we definitely have a queen doing the laying – but who?

I did see what I think was a queen bee – she looked different to the others and was certainly not a drone – wandering about over the comb, sticking her back end into vacant cells, pausing, then moving on – all of which is queen behaviour as far as I’m aware, but she wasn’t marked with a red spot.

I’m wondering if the swarm decided to re-queen all by themselves while we were away, or even before we went. Today was the first physical inspection we’ve made since spring as the weather has been pretty chilly and damp. The only opening of the hive we’ve done up till now has been to feed the girls.

I didn’t see anything that looked like an open queen cell on any of the comb or frames, so if they have re-queened, then I’ve got no idea where she came from! There was one large cell, open, on one frame, but it looked more like a drone cell than a queen cell. Confused? I certainly am!

So there you have it, bees don’t seem to need us humans as much as we seem to need them. We are still eating last years honey – Alison makes the finest honey flapjack in the world with nothing more than oats, honey and juicy sultanas – yum!

Varroa counts are low, I saw only a few when I did my count today, and it worked out at less than one per day over the last couple of weeks. I’m extremely suspicious of this, but gave them a dose of icing sugar anyway – just to be on the safe side.

Cheers.

Bee Tragedy!

Norman — Sat, 09 Apr 2011 09:54:15 +0000

I’ve taken far too long to write this up, but I’ve been rather pissed off to be honest! We’ve lost one hive of bees since February. They were alive when I checked and I’ve been feeding them but I noticed that the feed wasn’t going down. Due to the weather, I couldn’t “go in” and see what was what in case I did them some harm with the cold and damp. I wish I had to be honest.

We had a warm day about three weeks ago, so I looked in, still food left, but no sound, no movement from within. Fearing the worst, I lifted the crown board a little to peek inside, nothing. Hmmm not good.

Given that it was reasonably warm, and that I had my cloth cover with me, I whipped off the crown board and I was met with a scene of carnage. Every single bee was dead and they were all lying in a heap on the floor. Why? They had starved to death.

Even with the fact that there was lots of feed on hand and close by, it seems as if they refused to leave the cluster and eat. This happens (I’ve been told by Peter) and there’s not a lot we can do. The cold, damp weather may have kept them in the cluster and they would not have moved away to feed – starving themselves to death while there was plenty of food all around.

Every cell in each of the 5 frames that had drawn out last year was totally clean. A couple of bees had died “head down” in the cells looking for the last of the stores that they had accumulated. I find this all quite distressing – so much food, so many deaths.

This was our first swarmed hive too, with a new queen. We still have lots of bees in the original hive, but Queenie is a “red spot” and is now in her fourth year, so we may need to re-queen to keep that hive going strongly. We don’t want her to be laying drones all the time – that’s fatal for a colony.

Cheers.

Domino is gone

Norman — Thu, 17 Mar 2011 20:19:02 +0000

Domino, our California Kingsnake was put to sleep tonight at 19:10.

She’s been in the family for about 13 or 14 years with never a problem, well, except for what you can see below – she got me one night and had my finger down her throat before we could do anything about it. Alison decided to take a picture rather than get her off me!

It eventually took over an hour and a half, and judicious use of January ice cold water (as advised by the RSPCA) to get her off, safely.

Domino Bites Norm! (January 2003)
Click for large sized image.

What you may not be able to make out in the picture above is the fact that she has effectively tied my wrists together – she had quite a grip, I can tell you. Mind you, she is a constrictor, so she would be strong in that department!

Recently, she developed a lump on her tail. We took her to the vet but after a course of treatment, there was no response. She hasn’t eaten now for two weeks – not a problem in itself, she’s gone much longer without in the past, but she has lost a lot of her muscle tone and her spine was showing quite prominently. Even the vet was surprised at how much she had gone downhill since she last saw her.

We shall miss her.

Cheers.

Japanese Tragedy

Norman — Thu, 17 Mar 2011 14:22:51 +0000

Earthquake, tsunami and nuclear problems. Like they don’t have enough trouble with just one of these?

And as usual, the religious fuck-wits are coming out of the woodwork with their completely ridiculous claims that it’s all “someone’s” fault – be that someone the gays, women (it’s usually one of these groups to blame for everything according to the fundies!) or someone else.

Mind you, the Daily Mail isn’t much better, reporting on a astrologist (anagram of fuck-wit) who “predicted” the carnage. Read more, if you haven’t yet eaten your lunch, here

A woman cries her eyes out in the ruined town of Natori, Miyagi Prefecture. Click for more pictures, full size.

We all need to help out, so give money (scroll down to the flags), praying doesn’t work – it has been proved – except to make religious nutters think that they have done something useful – so unclasp your hands and get them into your pockets!

The Frogs Are Back!

Norman — Fri, 25 Feb 2011 13:14:23 +0000

Yesterday, 24th February 2011, was about the first reasonably warm day of the year. The pond and paths around our house are now filled with frogs. It must be spring!

The Bees are alive!

Norman — Thu, 10 Feb 2011 12:15:25 +0000

On Sunday 6th February I hefted the hives as usual. The new hive felt a bit on the light side so I whipped the lid off (just the lid) to take a peek – it was quite mild or I wouldn’t have disturbed them.

Lifting the card I use to block the ventilation from passing through the cluster, I saw lots of live bees! Yippee! Things are looking good for a second successful winter, and this year was a cold one!

Anyway, we have a pile of honey from the natural comb from last year so I decided to get some on as the girls felt a bit light to me (this is our first ever “home grown” swarm and as yet are still only on 5 brood frames). I’ve added a super and put a feeder in with some of their honey. A dribble went down the central hole to get them interested – just in case.

Hopefully the weather will stay on the mild side and they’ll be able to get up and feed. I’d rather feed them honey than fondant – that needs a lot of water to dilute just so that they can evaporate the water again! Doesn’t make sense to me.

While I was pouring honey into the feeder, a couple of bees flew out of our main hive, and took a look at me before nipping back inside the hive. Looks like we may have managed to get two hives through the winter.

This is our second year keeping bees, our second winter and it seems we still have beginner’s luck!

We may need to re-queen our main hive this year too, she’s red and now at least 3 years old. I need advice …

Queens are marked for the year they were born:

White Yellow Red Green Blue – calendar years 1 and 6 are white, 2 and 7 are yellow and so on, up to 5 and 0 which are blue. This year, 2011, is a white year and our queen is red, so she’s most likely to be 2008. Apparently after 3 years there is a strong possibility that she is running out of stored sperm and may start laying nothing but drones. Fatal for a hive.

Cheers.

Signs Of Life?

Norman — Tue, 25 Jan 2011 15:15:58 +0000

I was out this Sunday hefting the hive. It’s feeling a little lighter than I remember, so I suspect the girls are tucking in to the stores now. I heard from Peter, about a fortnight ago, that his girls have started laying already so it’s looking good for Spring.

As I was hefting, one solitary bee flew out from the mouse guard and buzzed around me to have a look, then vanished back into the hive. This is good because it means that we have at least one bee that survived the winter!

I’ve got some honey from the Autumn crop all ready to feed them with, if they need it. Much better than sugar syrup if you ask me.

Cheers.

ExecutingPackaged Code Over a Database Link

Norman — Tue, 18 Jan 2011 15:40:49 +0000

Ever wondered how you can call a packaged procedure (or function) which resides at the far end of a database link? Wonder no more!
begin package_name.Procedure_name@db_link_name(parameter, parameter, ...); end;
Calling a function is just as easy:
declare vResult number(10); begin vResult := package_name.Function_name@db_link_name(parameter, parameter, ...); end;

Unfortunately, you don’t appear to be able to do this sort of thing via a synonym:

create or replace synonym X for package_name@db_link_name; begin X.Procedure_name(parameter, parameter, ...); end;
You get ORA-00904: “SQLTEST”.”TEST”: invalid identifier instead of a result.

Update: 20 January 2011:
If you prefix the remote object name with the remote object owner, regardless of the fact that the database link is connecting to that schema anyway, then you can subsequently call the function, procedure or packaged code using the synonym. Result – thanks to Oracle Support.

create or replace synonym X for owner.package_name@db_link_name; begin X.Procedure_name(parameter, parameter, ...); end;
It “just” works.

Cheers.

What Do We Need, Banks or Hospitals?

Norman — Tue, 18 Jan 2011 09:17:49 +0000

The government’s wonderful new system for the NHS starts tomorrow. GP’s will take over running things where the Primary Care Trusts used to and Primary Care Trusts are to vanish. If any hospital fails under the new system, there will be no tax-payer funded bale outs.

Strange that, they baled out the banks and have been shafted in return as bankers returned to their old ways – the ways that all political parties said would never happen again – and stuck two fingers up at the tax-payer and the government. But there’s no such luxury for a hospital. Hmm.

Nice to know where their priorities lie then!

As I always say, voting should be mandatory and there must be an option on every ballot paper entitled “none of the above”. Mind you, I also have said for years don’t vote, the government will get in!

Cheers.

Very Sticky Sunday!

Norman — Sun, 10 Oct 2010 20:06:00 +0000

Today, Alison and I took our lives in our hands and extracted the honey from 20 full frames. In all, we have now got a kitchen full of about 70 lbs of fresh garden honey courtesy of our hard working lady friends. The day started with the de-capping of the combs, four of, and then putting those into the centrifuge – a giant salad spinner of sorts. After 60 turns of the handle, we open up and turn the combs around and give the other side 60 turns as well. repeat for the other 16 frames.

Of course, as soon as you get a good extraction, you have to empty the centrifuge through a double mesh filter and into a bucket (sterile of course). This filtering gets rid of various lumps of bees wax, assorted bee parts – legs, heads, the odd wing etc and leaves you with fresh, clean and very tasty honey ready for bottling.

It’s hard work, but not as hard as cleaning up afterwards and getting tiny lumps of sticky bees wax off the floor of the kitchen! The empty frames have all bee put back into a super and put on top of the two hives. While the centrifuge gets most of the honey out, it leaves everything sticky. The bees, on the other hand, will clean up everything and take it down into the brood box for winter stores. In return for this bounty, we get the combs back clean and dry and ready for storage.

We give the girls back the honey because it’s theirs after all, it would be cruel not to let them have some. We have kept back about 10 lbs of honey – in raw natural comb – for the girls to feed on over the preparation for winter and in early parts of the year when they need feeding ready to being. No sugar syrup this year for our girls!

Cheers.

I’m Seriously Offended …

Norman — Mon, 04 Oct 2010 12:30:22 +0000

It seems that this new government is off it’s bloody head. A new law has been introduced that means, in essence, that if someone tells a joke in the office, and someone else is offended by it, they can sue, even (get this) if the offended person didn’t actually hear the joke in question!

WTF is the government doing for crying out loud? Have they not learned from the mistakes of the last Tory government (aka selfish b’stards) – led by the “lovely” Maggie T. – who were the ones that introduced the American style “sue everyone, I’m not to blame” culture, that it just leads to complete nonsense? It appears not.

I’m seriously offended by this new rule – who do I sue?

Cheers.

The Final Harvest Is In…

Norman — Mon, 04 Oct 2010 12:25:05 +0000

Saturday 2nd October was a fairly nice day given the weather we’ve been having recently, so we managed – at long last – to get the final harvest of honey done. We managed to take another 9 full frames and a bucket full of raw natural comb also completely full and capped off. This was the space I inadvertently left when I removed the initial three frames. The girls went mad and filled the gaps.

The moral learned was never ever give them any free space because they will fill it up!

Anyway, we have now got the supers off, the queen excluder off, the eke on, the apiguard (anti varroa) on and we are just about almost nearly ready for winter – just the mouse guards to go.

We intend to keep the honey in the bucket to use as winter feed rather than supplying them with syrup that they have to work on to convert into “honey”. I suspect giving them back their own honey is far batter for them in the long run.

Peter’s extractor was delivered by Helen last night (Sunday) so next Sunday will hopefully be the day that the kitchen gets extremely sticky as we attempt to extract honey from all 19 frames. Watch this space for details.

Next time, I need to make sure I get hold of a couple of Porter Bee Escapes to keep the girls from the capped off honey stores – it’s a lot easier to extract the filled frames when you don’t have to deal with a swarm of slightly irritated bees!

Cheers.

The Pope Is Coming – Lock Up Your Kids!

Norman — Thu, 16 Sep 2010 07:42:59 +0000

The pope arrives in the UK today. Personally, I think he should be arrested. As the head of an organisation that is riven with child molesting staff he should be arrested. As the man in charge of covering up the atrocities (while a Cardinal) he should be arrested. As God’s disciple on Earth – he’s a fraud!

Why are the British Tax Payers, at a time when cut-backs are “required” paying for the head of the Vatican “state” to pay a “state” visit to the UK, forced to pay huge sums, running into multi-millions of pounds to provide security for this man?

If he really is god’s disciple on Earth then god will prevent harm coming to him, surely?
If someone does harm him, then surely it was god’s will, and we lowly Earthlings shouldn’t even begin to attempt to subvert god’s will?

So, regardless, we don’t need and shouldn’t have to pay a single penny for security on this visit. Just a thought!

Anyway, you can tell how much real faith all churches and mosques etc have by the fact that they all have lightning conductors on the highest points. Go figure.

And finally, as the pope is arriving in the UK via Scotland, here’s a question:

Why is it, in public toilets in Glasgow, that people write “Fuck the pope” on the walls? Because they can’t be bothered to write “Fuck the moderator of the general assembly of the church of Scotland” (Billy Connolly)

Cheers.

Oracle Change The Patching Process From 11.2.0.2

Norman — Mon, 13 Sep 2010 08:19:43 +0000

Oracle have made subtle changes to the way that they advise you to patch an installation from the release of 11.2.0.2. The full gory details can be found here (My Oracle Support link – support contract required) but in brief:

You are now advised to perform an out-of-line patch, however, you can do an in-line patch if you are short of space. However, this will delete the existing installation. Beware & take copious backups!
The patch kit is a full installation – so you get the latest patch kit and install it instead of getting the base release and then patching it.
Grid components must be patched out-of-line.

Cheers.

Bee Latest.

Norman — Sun, 05 Sep 2010 19:41:36 +0000

The girls were checked again today and all is well. Varroa counts is nil on both hives – that worries me a tad! But I did give them a good dusting of icing sugar last week – so maybe it works after all!Anyway, I managed to swap out 5 old brood frames from the new hive – the bees were not using those frames and on inspection, it’s easy to see why – old dirty wax, some dead brood in the combs so they have gone, destroyed. I thought about cleaning them up and re-waxing them, but you never know what has been in there.

These were frames from Peter’s hive out in the woods near Leeds and he did say that the swarm in there originally had died for some unknown reason. Before he got a chance to clean up, a new swarm moved in and we were the lucky recipients of said new swarm last year.

They are starting to store quite a lot of honey on the brood frames, but there are still larvae so the new queen is doing her stuff – I still have not found her yet, but when I do, she will have to be marked – something new for us to learn!

The original hive is also doing well. They are out and about quite a lot at the moment making the most of the end of season sunshine and, of course, the honey flow from Himalayan Balsam.

After we took a super of frames off this hive last week, they started to fill in every available space with comb and honey, so I put a new super on today with 9 brand spanking new frames. I’m not expecting them to draw out and fill all nine, but at least they have somewhere to store all that lovely nectar!

All being well, we will be able to feed them honey, of their own making, to keep them happy over winter.

Talking of winter, it doesn’t seem like it was this time last year when we were just getting ready for our first over wintering of the bees. Here we are again, getting ready to over winter two hives, one strong and the other, not as large, but doing quite well so far. Other than in the stores department!

Cheers.

It’s Harvest Time!

Norman — Tue, 31 Aug 2010 18:54:57 +0000

The girls have been busy and today, we have harvested another 9 frames of fully capped honeycomb to add to the three we took a couple of weeks back. Unfortunately … … I neglected to replace the three full frames I took out last time with new ones. The picture below shows what the bees will do if left to their own devices and when they find a big space in the hive. The entire space has been filled with natural comb and filled and capped all in a couple of weeks.

Busy or what? Anyway, here’s the picture.

The space for three frames has been filled.

Unfortunately, as you can tell from the picture above, the natural comb has been broken when we split the super off the top (there’s another super underneath – that’s what you are looking at) so there’s a bit of loose honey that the girls will have to tidy away quickly now – in case of robbers, and bloody wasps, of which there are many this year.

We still have six frames, capped off, to harvest plus the mess above. I have a feeling I may be using the natural comb as feed for the winter.

At present, the girls are heavily into Himalayan Balsam which is good for them, but not so good for our native environment. This stuff is an alien and takes over quite rapidly when it arrives – but the bees love it and it flowers late into the season keeping them on good natural honey which, with a bit of luck, will be more than enough to keep them through winter without having to be fed on sugar syrup.

We may have to let them have their heads, as it were, and give them a few new super frames to fill up with winter stores. I’d much rather my bees fed on their natural food than on sugary substitutes that we supply. It makes sense to me.

Varroa counts were up a little – we have about 16 on the tray today, so while not bad, it’s not good to have any, so the hive got a good dusting of icing sugar. With the honey still on the hive, we can’t use any of the chemical (thymol) based stuff, but once we get the last few frames off the hive, there will be some going in. We need to get the girls healthy in time for winter.

The other hive is still getting established. The girls in there have managed to use about 5 full frames to get the queen laying and they have pollen and honey – capped off now – aplenty. Not enough for the winter though, not by a long way. The remainder of the frames have not bee touched. They don’t seem to want to do much wax works inside the hive this lot.

They haven’t begun to create or lay, any drone comb either, but with it being September (tomorrow) I doubt we’ll see any of that from them now, it’s getting on for drone expulsion time, so any boys in the hive better enjoy life while they can, it’s soon to be all over!

As an aside to the main posting here, it’s a hard life being a boy bee – in case you didn’t know, drones are the boys and they exist for one thing and one thing only, sex! (How much does that resemble us Humans?) They sit around in the hive all day doing nothing, providing nothing and eating stores, waiting for a virgin queen.

They also hang about in gangs at a drone meeting place, where a virgin queen will come and mate with as many of the boys as she can. This is her only chance of mating so she takes full advantage!

For the boys this is it! Sex, then death, by having your bits ripped off! Still they seem to like it! If the boys manage to avoid being killed off in this way, come September, the girls will either evict them from the hive, or sting them to death. There’s no place for boy bees in the hive over winter.

At least the wax moth problem I mentioned last time has been sorted, no more signs of any, which is a good thing. And even better, the varroa count was one single solitary mite.

This new hive is going to need quite a bit of feeding before winter. And I still haven’t found our new queen yet! I know she’s in there because there are lots of lovely larvae in the brood frames, but she’s keeping her head down and hiding from us.

Peter, if you are reading this, thanks! We couldn’t have managed without your help and advice.

Cheers.

To Flashback, Or Not To Flashback. Are You Sure About That?

Norman — Wed, 25 Aug 2010 14:13:19 +0000

John Hallas has an interesting blog post here about how a system running flashback can become a system not running flashback – and without you knowing!

Enjoy.

Cheers.

The AA Insurance Company “Scam”.

Norman — Mon, 16 Aug 2010 08:28:43 +0000

My wife recently changed from her old insurance company to the AA as they gave her a much better quote for herself and me as named driver. However, it soon turned out to be not quite as attractive as it first seemed. Shortly after sending in the forms and money Alison received a call from the AA saying that there was a problem with her application and could she call a certain number to discuss. She duly did so and was told that as I had a claim against me, that they were increasing the premium by approximately £60 per year.

This was interesting as I don’t have any claims against me at all, however, they assured my wife that this claim against me took place in 2007. I was puzzled and decided to check.

I spoke first with the AA who assured me that there was indeed a claim on the database and so Alison had no choice but to pay up the additional premium on her police (or drop me as named driver). I got as much information as I could from the telephone operator – who was very helpful just powerless to deviate from the rules she had to work under – and eventually asked for her manager. S/he wasn’t there but promised to call me back. So far, no call back!

I then spoke to my own insurance company, Sainsburys who I highly recommend and they explained everything about this claim. I was informed that, on their database it is listed as “no fault, no claim” as it was simply noting that I had advised my insurance company of an accident in which I had been hit, while stationery, by another car. The other driver immediately admitted full liability and the whole matter was resolved with no claims being made against me and my own insurance company merely being informed of the facts.

So, a non-claim against me turned into a claim against me on the AA’s database and they flatly refuse to change it, or even speak to my own insurance company for clarification.

Next time you think you need the AA, try here or here instead – the AA’s own claims are probably as dubious as the one they think has been logged against me.

It’s not the first time that I’ve had trouble with this organisation, they gave me a load of grief years back when I was a motorcyclist and I promised myself never to use them. I was happy with the RAC for years – until Alison had some grief with them once and changed our “contract”.

I stand by my previous decision never to become a customer of the AA. (Unless they sort their act out pretty sharpish and refund our additional expense.)

Would I recommend anyone take out insurance with the AA? In a word, no. (Other opinions are available, but they are wrong! )

Having said all that, I do appreciate the fact that the AA are not actually an insurance company, just a broker (or are they?) for a real insurance company located somewhere in Gibraltar and that the AA probably have to follow the rules laid down by said off-shore insurance company but does anyone actually think about these rules? I mean, how on Earth can a “no claim” be a “claim against”? Sigh.

Cheers.

How To Gather Table Stats Effectively

Norman — Mon, 16 Aug 2010 07:36:06 +0000

Greg Rahn wrote a pretty good article back in 2008 – it still applies today.

Cheers.

The First Honey Crop

Norman — Wed, 04 Aug 2010 12:12:15 +0000

Been a while since I wrote, so here’s a brief update on what’s happened since last time. The artificial swarm was carried out and we watched hopefully for a hatching new queen. We didn’t see her, but we have evidence that she is indeed there and laying. The hive is filling with bees and we have larvae in the cells. The vast majority are worker cells and there are very few drone cells – so it’s looking good for a new laying queen as opposed to a laying worker.

We were in Sardinia for two weeks and on our return, it was impossible to separate the top (two) supers from the main hive. The girls had glued everything together. It didn’t help that I’d lost the hive tool somewhere and it simply seems to have vanished from the planet. I suspect it was beamed up at some point between the hive and the garage!

Anyway, a new hive tool was purchased, and I went in to the hive last night when the weather was quite mild. The new tool was an immense help in separating the two supers and there is currently 16 frames of honey. Three were fully capped off, so I’ve removed those for later. I’ll be putting another three empty frames in to keep the girls busy.

Between the supers, they had built comb from the top of the lower super to the bottom of the frames in the upper one, that’s what made it difficult to split the two. After last night when I scraped off what was bridging the two supers, we have a bowl of wax + honey to enjoy. A bit more work needs doing on this matter, but the girls were getting grumpy and I’d refilled the smoker three times by then, so I decided to let them have some peace and quiet.

The second hive had a plague of wax moth caterpillar, well two of the b*ggers, so I got rid of those. One brood frame was pretty much trashed but it was an old one I fancied replacing as the girls weren’t using it yet – so it’s now gone. I ditched the wax as well it was looking pretty black. (It was one of the original frames we inherited when we got our first swarm from Peter.)

When I checked for varroa, that nasty little mite, the counts were still low. About 4 weeks ago (I know, far too long!) I had 16 on the second hive, but after a fairly liberal sprinkling of icing sugar, we are down to about 3 mites over the 4 week period. The main hive had 1 mite over the same period. I’ve also done a drone cull in the second hive (where the frame came from the main hive – don’t get confused like me!) and there were a few mites present in the drone comb, but nothing spectacular. Yet!

I’ll be keeping on top of the mites on a weekly basis now that the holidays are over.

Cheers.

We Have A New Queen Coming!

Norman — Mon, 31 May 2010 11:42:05 +0000

It was been checking time again yesterday and of the three queen cups I found last week, one has a grub in it. Looks like we need to consider doing an artificial swarm for real this time.

The first super we put on is now in the process of being capped off. The honey is almost ready. I’m leaving it for the girls to completely cap off and then, Alison gets to do what she’s been looking forward to since we started on the Bee Keeping course, extracting her own honey! I foresee a very sticky kitchen any time soon.

The second super is starting to be filled as well. They have drawn out 7 of the 8 frames and have simply left the last one untouched. Not bad for a week though.

Queenie is still around, I saw her yesterday. Wandering about and laying eggs as usual.

Just How Busy Can a Bee Get?

Norman — Mon, 24 May 2010 19:17:29 +0000

On Saturday I went to check on the girls again. In the seven days since my last check, they’ve been busy. How busy? Well, last week I gave them a pair of brand new super frames to fill up the super. They had 6 already (or was it 7? I must go and count next time I see them!) and they had just started to draw out the comb. Well, 7 days later, we not only have a complete set of super frames, fully drawn out, but the whole lot is just about full (on both sides!) with brand new honey. Once they get it evaporated down to the correct water content, they will cap it off.

I suspected something was up when I lifted off the super to do my weekly checks, I could hardly lift it! Anyway, I’ve kept them busy by giving them a new super on top of the existing one, with a compete set of empty frames. Wonder what I’ll find next weekend?

Downstairs in the brood box, Queenie has been busy and there are “millions” of bees milling about. Plenty of stores – honey and pollen – stashed away in the brood frames. The drone comb that we cut out last week (to try and prevent varroa infestation) has been completely rebuilt with new, fresh beeswax. No eggs yet.

We also have three potential queen cups. No eggs (not that I can see them with my eyesight – glasses on or not!) or larva yet, but I’ll be keeping an eye out, just in case.

And the varroa count. Well, after 7 days, I counted what was on the varroa tray under the hive – a bit of chalk-brood, and one, single, solitary, all by itself, mite. One!

This is good news, but worrying at the same time. I can’t believe that our hive doesn’t seem to be a hot bed of varroa like almost all the others. There were some in the drone comb we removed last week, but not many. I haven’t see any of the bees with mites, but then again, it’s difficult to tell with so many of them knocking about.

How many times have I been stung when I dismantle their house? So far this year, none! They are very docile at the moment, I’d personally be going off my head if someone kept breaking into my house and moving the furniture all the time!

Cheers.

Ted Hooper – RIP.

Norman — Thu, 20 May 2010 20:01:52 +0000

Ted Hooper has died at the age of 91. He was most famous for the Bee Keeping book Guide To Bees And Honey.

Some links to Ted:

Wikipedia

British Bee Keeping Association – Obituary.

Times Online – Obituary.

We Have A Queen – And She’s Red!

Norman — Thu, 20 May 2010 19:54:45 +0000

On Saturday, Peter came over because we (Alison and I) thought we had noticed a few queen cells in the hive. Drastic action could have been required if this was the case with an artificial swarm being uppermost on the list. As it turned out, we were wrong and what we had seen was a number of drone cells, not queen cells. Ah well, at least we know now!

While inside the hive, we found the queen – and she was marked, with a red dot. This makes her either 2003 (highly unlikely) or 2008 (most likely) which does imply that our swarm of bees used to belong to someone else!

Anyway, a full investigation showed lots of new brood, plenty drones and workers (phew!) and while we were inspecting, the queen laid an egg right in front of us.

They have also drawn out about 6 super frames from bare foundation since the start of the year. There is even honey in some of the super combs already.

They are still surprisingly docile, we basically destroyed their house, and they were not bothered at all.

We now have to check them every 7 days to make sure that they don’t start getting ideas about leaving home. Mind you, we have a new hive sitting right next door to the existing one, so there’s always a chance that they will move in there.

Cheers.

The Election Is Coming!

Norman — Sat, 17 Apr 2010 17:32:19 +0000

Oh joy! Politics. How lovely. Ok, here’s what I think of the election:

They don’t give a shit about keeping promises. Regardless of what your local MP (or wannabe MP) tells you he will do for you, once (if) s/he gets elected, s/he will have to follow the party whip – so why do they bother telling us lies all the time.
They still have an expenses system that gives them the right to excess. If I took a job in London, for example, I would not be getting second homes allowances, nor any of the rest of the stuff that they provide themselves with. It’s a con. They know what the job entails and where they have to work from, so deal with it. if you don’t like working to the same expenses rules as everyone else in the country, don’t apply for a job as an MP. Simple.
Voting should be mandatory in the UK. However, see next bullet point.
Every voting paper should have one additional ‘candidate’ at the bottom – None Of The Above – that would give the politicians a better idea of exactly what we think of them. (None Of The Above courtesy of Brewster’s Millions an old film with Richard Prior.)
And finally – for now – remember this if nothing else. You can always tell when a politician is lying, because his lips move!

Cheers.

Fly Monarch? I Don’t Think So!

Norman — Wed, 14 Apr 2010 15:39:47 +0000

On Saturday the 10th of April, at 20:30 our journey home from Sharm El Sheikh began on a Monarch Airlines Airbus A300. We were sitting in row 15, seats A and B – if it makes a difference. 15A being the window seat and 15B being the middle of three.

For someone over about 5’6″ legroom is a problem and as Alison is 5’8″ and I’m 6′ we both were in trouble before we left! I had to request that the people in front of me didn’t attempt to put their seats back as they would crush my knees if they did – thankfully, they obliged even though that had to have their 2 year old on their knees. Not an ideal way to start a long (approx 6 hours) flight back to the UK.

The window seat has a big metal bracket screwed to the floor restricting the available space for feet under the chair in front. The aisle seat has a big “box” of some sort that also performs the same task. Only the middle seat – where Alison was – has any decent amount of space for your feet – although the middle seat is always seemingly narrower than the others.

So, an auspicious start to the nightmare ahead – no leg room and nowhere to put your feet either.

Did I mention also that the flight was short staffed as two Cabin Crew had failed to arrive for work? Well, all compliments to the girls who had arrived – they did a great job. Credit where it is due.

On the ground, the interior of the plane was far too hot. We were told that after take off the air conditioning would come on and reduce the temperature. Take off couldn’t come too soon!

Anyway, we soon set off, on time (ok, 5 minutes late!) and immediately we hit a problem. The temperature inside the cabin soared into the heights. People all around us were fanning themselves with the in-flight magazine, sick bags or anything else they could find. The captain was asked to turn it down. It made no difference.

Did I mention that we had been given details in the safety briefing on where to find the exercises we really should do to keep ourselves from the scourge of Deep Vein Thrombosis (DVT)? Well, reading about the exercises was about all we could manage – there’s no room for your feet under the chair when sitting still so you have absolutely no chance of “extending your leg and waggling your foot around and around” – once sat down in a Monarch Plane, you simply cannot move.

Once safely at height, the cabin crew sold us drinks. We again asked to have the heating turned off and they asked the Captain to do so. He did. It made no difference. Maybe third time lucky?

If I remember correctly, two tiny bottles of Brandy, two tiny tins of Ginger, and two tiny bottles of white wine (unchilled for crying out loud!) and a tiny box of Pringles cost us about £17.00. Ouch! Still anything to take the mind away from the uncomfortable situation we find ourselves in has to be worth that, doesn’t it?

Shortly afterwards, and it’s still far too hot, it’s dinner time. Sausage, mash and two vegetables. Sounds good doesn’t it? Unfortunately, airline food all smells the same – it must be the altitude and reduced cabin pressure – and it smells (to me) like vomit. I cannot eat airline food. Dinner for me was a brandy & ginger and a bottle of wine. I tried to chill it in a cup with two ice cubes in it – I failed. The heat in the cabin rendered the ice cubes into water in about 5 minutes. Sigh.

We ask the crew to turn the heating down, or perchance, off completely. They promise to do so. It doesn’t work. Aaargh!

The entertainment is a rerun of some old crappy TV programs, which if memory serves me, started with something dire, was followed by a program about the Beatles, that then Progressed into a Tribute To Heath Leger followed by some other ancient drivel from the TV and got worse after that. It’s still too bloody hot!

It makes no difference anyway, the headphones supplied (excuse me, sold to us for £2.50 each) don’t work. Well, they do work, but the holes in the seat to plug them in are so close to your thighs that you can either have the headphones on or you can sit in the seat – but not both!

I suspect that after a few passenger plus headphones plugged in interactions, the sockets are trashed. No sound. It was the same on the way out as well. Thankfully, the TVs are small and so far away from where I’m sitting that watching is almost impossible anyway.

Later that same day …. it’s almost midnight. Guess what, it’s cooled down a lot now. The Captain finally managed to turn the heating off/down and we are almost comfortable – temperature wise I mean. The cabin comfort levels are non-existent I’m afraid. Everyone (it seems) is stiff and unable to move. I didn’t remember anyone putting their seats back to try and get a bit more comfortable – maybe the sound of cracking knee caps from behind put them off?

The cabin is full of people wanting desperately to go to sleep. The staff will sell you “comfort kits” consisting of a blanket, a blow up neck pillow and a mask. Hmm.

You will need the mask because even though it’s midnight (Egyptian Time) the Cabin staff are now selling Duty Free crap goods from the trolley at prices that appear way too high to be anything but Duty Free. (Other opinions are available!) The Cabin lights are turned up to full and other than take-off, they haven’t been anything but.

Now, the blow up pillow are a great idea. In principle that is. The idea is that you blow them up and place around your neck so that you can lean back in your chair (yeah right!) and have a nice comfortable place to rest your weary head as you drift off into a deep and comfortable sleep.

Reality has a nasty habit of changing things! The seats that we are supplied with are way too low in the head rest department, and any attempt to sleep with your head back results in your head falling over the back of the seat and irritating the people behind you! There is nothing at all to rest your head on! The seats in my car have more height to them than the ones in this plane! Monarch are the only airline I know that have such low seats.

Trying to rest your nice comfy pillow on the seat simply won’t work – you’ve been sold a pup – as the people in front of me discovered when they tried! At least the mask comes in handy as the Duty Free trolley is still slowly doing the rounds and the cabin is lit up at full brightness.

Trying to get to sleep I rest my head on the back of the chair in front of me. It doesn’t work. Any time anyone in any of the three seats in front of me moves – or breathes heavily – the vibrations transfer straight into my head.

I’m in the window seat (15A) and I think I could be able to lean my head against the window or the wall. How sadly mistaken was I? I start to nod off and my head falls backwards into the window recess, twisting my neck at an unnatural angle and wakes me up. I try again. Same result. I swear! Twice.

I suppose I forgot to mention the fact that the seats and the windows don’t line up? Well, they don’t. Some people can see out of two windows (ok, one and a bit) while some people can’t see out at all – even though they have a “window” seat. Hmm.

Ok, sleep is going to be difficult I can tell. But I try again, this time I must have drifted off for a while as I wake up to find the lights in the cabin have been dimmed. I look at my watch – why am I awake? Oh yes, I fell into the window again – we still have hours and hours to go yet.

I give up on sleep. I look out the window – there’s nothing on TV and anyway, my earphones don’t bloody work! (See above!) The view down to ground is quite pleasant – it’s a clear night and you can see for miles. Looking up, I can see even further as I can see stars! Oh well, that’s the good bits done then, back to the flight!

I attempt another attack on my complete exhaustion by leaning my head on the seat in front again. Alison – herself knackered – tries to sleep by leaning on my back. I think I drop off but the liquid lunch seems to be causing weird dreams (or it could just be the surreal reality of this flight!) and I’m not sure if I’m awake or asleep or what. Anyway, it doesn’t work and Alison gets off of me pretty soon. I’m awake again.

Alison has given up on sleeping now and has been trying to do Sudoku on my Nintendo DS but she soon gives up on that and starts chatting to the man in 15C. He turns out to be on permanent holiday since being laid off from his job as a civil engineering something or other! He was responsible for there Leeds City Tram project that the government canned. They chat for ages while I doze – my head still jammed onto the seat in front. I’ll have a textured forehead when we eventually get back to Manchester!

There’s a famous person on this flight as well. David someone or other. He plays the caretaker in the Harry Potter movies. I didn’t recognise him when he was chatting to us on our bus on the way out – or on the way back. Nice man. He’s knackered as well and seems to have given up on sleep and sitting down. He stood up for the best part of the flight.

Many people, when I look around, are doing the same. Standing. It’s the only thing to do. I suspect the number of numb bums on this flight is approaching 100% – mine certainly is.

I try to sleep. I doze mostly and I seem to remember when I could actually feel my feet. Thanks to that metal bracket (and the sloping curved sides of the plane) I’m sitting with my legs and feet at what can only be called “an unnatural angle” and I’m really uncomfortable. The man in front is in a similar situation and is complaining about his back giving him grief. I suspect others have a similar mindset.

Suddenly I awake from my doze. The plane has throttled back and dumped it’s nose into a downward direction. We must be descending into Manchester. I look at my watch and I can’t figure it out, we are early!

Half an hour later we are down and off the plan. What a relief. That flight was a nightmare and I’m surprised that Monarch are allowed to put passengers in a situation where they have no leg room and no foot room while sitting on a seat with no ability to lean back and rest their head. I’m not surprised people get DVT when flying.

Sometimes the only thing that spoils the joy of flight is being on a plane.

Shag Fest!

Norman — Mon, 22 Mar 2010 15:25:32 +0000

There’s a mass orgy taking place in our pond this weekend. The frogs are hard at it again – it must be springtime!

The first spawn I found was actually in the filtration housing, one batch of spawn plus 38 frogs – 36 coupling and two single males looking for action.

I hefted all the above into the main pond itself, and left them to get on with it. Yesterday afternoon, the spawning had begun in earnest. This morning they are still laying.

It’s a shame that we have fewer frogs in the pond than we had on the last two years, plus those damned Koi haven’t died yet so any tadpoles that do hatch, will eventually be eaten. I saved some last year by a transfer to the small pond in the back garden – but it’s far too small to cope with the expected numbers of tadpoles. Sigh!

Cheers.

I Was Peed On By A Frog This Morning!

Norman — Thu, 18 Mar 2010 15:39:18 +0000

What more can I say?

I was leaving home, for work, this morning at 06:55 and when I opened the garage door (from the inside) a frog was sitting there, right slap bang in the middle of the doorway. She, for it was a girl frog, just sat looking at me.

Obviously, if I was to drive over her, she wouldn’t be too happy (neither would I, I like frogs) so I picked her up to take her to the pond.

She wasn’t happy about that at all. She peed all over my hands. Thanks girlie! Frog pee is cold by the way, in case you are interested!

Anyway, I took her up to the pond where rather a lot of other frogs are waiting for the right moment for a frog orgy and popped her in.

I didn’t kiss her though!

Cheers.

The Bees are Flying Again!

Norman — Thu, 18 Mar 2010 15:34:05 +0000

On Sunday (14th March) I went to check on the bees. Hefting the hive was a tad tricky as they were all out and about flying around and generally being bees. The weather had taken a bit of a warm turn (relative to recent months that is) and they took advantage.

A closer inspection showed a good few had been “hopping” over the hedge to a crocus ‘plantation’ not 10 feet from the hive and were covered in shocking yellow pollen. This is a good sign.

I decided to take the mouse guard off and let them clear out the bodies of those bees that didn’t make it through the winter. They do it so much better than me after all!

It looks like we may have managed to keep our bees alive through their first winter. Thankfully!

Cheers.

How Much Does it Cost to Keep Politicians Honest?

Norman — Thu, 11 Feb 2010 07:56:37 +0000

£6.5 million per year it would appear.

A new Parliamentary Standards Authority has been set up to monitor our beloved politicians and to keep them honest. According to Radio 4 today, this has an annual budget of £6.5 million and will employ 80 people.

Is it just me, or is it a tad strange that the people we elect to run (ruin?) the country should need so much watching to keep them honest?

Someone mentioned on the news last week, when discussing Grodon Brown’s proposal for the new voting system, that we should have an option that says “none of the above” – well, I’ve been saying that ever since I first watched Brewster’s Millions many years ago.

In my opinion, voting should be mandatory – like Australia – but every ballot paper should have the final option of “none of the above” and that way we can show politicians exactly what we think. Gordon’s new system doesn’t let us show our disapproval – funny that eh?

I suppose, looking on the bright side, 80 people will have a new job.

Cheers.

My Oracle Scripting Standards

Norman — Wed, 10 Feb 2010 15:05:53 +0000

Subject to unannounced changes of course, but …

All supplied scripts will be QA’d to ensure adherence to the following standards.

Any script that fails on one or more of the following rules will not be applied to any system. Instead it will be returned to the vendor for correction.

All vendor or internally supplied scripts must be able to be run – without error – in SQL*Plus.

We need this as a standard because of all the wonderful GUI tools out there in vendor land, Toad etc are fine – if you can guarantee that the person running the script has the same tool. SQL*Plus should be available in all Oracle installations. I much prefer Toad myself, but this is how it is.

I have seen (and run!) scripts in the past where semi-colons were missing off the end of statements. These had been tested and proved to work under some GUI or other that allowed this sort of thing – but come the day of reckoning, they failed in SQL*Plus.

All scripts must spool to .log.

This is so that we can see what errors occurred rather than hoping to spot them all as they zoom off the top of the screen at high speed!

The use of WHENEVER is not permitted.

It’s a pain when a script blows up at the first hurdle and then carries on, however, scripts which blow up and immediately vanish are even worse.

All scripts will explicitly set linesize, pagesize, trimspool etc.

No assumptions about how the client is set up shall be made. The vendor must explicitly specify all required options.

Commit and/or rollback is not permitted in any script.

Obvious really. If the vendor tests on a 6 row table and it all works, that’s fine. What if I have a 3.6 billion row table (and I do!) that, if updated, could blow away my UNDO tablespace? I don’t want some developer telling me it’s ok to commit in such an event, I’m the DBA so I decide.

Scripts must instead prompt the DBA to commit or rollback as appropriate after checking the log file (see above) as required.

Mixing DDL and DML is forbidden.

It is surprising how many vendors don’t know how Oracle works. The simply do not realise that DDL implicitly commits any outstanding transactions before it starts to CREATE or ALTER or DROP or whatever. It then commits again when done (if successful).

If you do see scripts from vendors that mix and match DML and DDL (and invariable compound the error by having a commit at the end!) then you should be on your guard for other problems – they don’t know what they are doing!

If it is impossible (!) to separate DML and DDL then it is imperative that all the DDL is executed first and all the DML executed last. This prevents the implicit commits from affecting the DML statements. However, it is advised that separate scripts are supplied for the DDL and DML parts.

Scripts that create stored procedure code must SHOW ERRORS.

It’s always nice when that well tested piece of code that creates a new package or whatever, fails to compile on my databases, but it doesn’t tell me why. Slipping a SHOW ERRORS in at the end of the CREATE OR REPLACE … is a nice little touch and keeps the DBA happy. The DBA is especially happy as the errors are logged in the spool file (see above) and can be sent straight back to the vendor for correction!

Do not drop temporary tables in the same script that created/used them.

This shouldn’t ever occur based on not mixing DDL and DML, but I have seen scripts which:

Create temp_table as select * from live_table;
DML the temp table to massage the data.
Drop live_table;
Create new live_table as select * from temp_table;
Drop temp_table;

Now, the obvious problem is this, it always fails at step 1. Steps 2 through 4 obviously also fail resulting in the loss of the live table and all it’s data. (10g recycle bin not withstanding!) and finally, we drop the temp table at step 5 losing all the data completely.

At this point, it’s back to the backups – you did take a backup didn’t you?

Dead Bees :-(

Norman — Fri, 29 Jan 2010 15:16:11 +0000

I hefted the hive the other evening – checking for the remaining supplies – it’s still heavy, so the girls should be fine for a while yet.

Unfortunately, when I peeked in through the mouse guard, there was a pile of dead bees. Not good. There are still lots of live ones, clustering on the combs to stay warm (and alive!) so that’s good. It’s just a shame that winter kills off so many of them.

The other thing is, being on our first season of actual bee keeping, we don’t know what to expect in winter. The last couple have been wet and miserable and that’s what kills bees, damp, not cold.

We shall see what happens soon. The queen should be up and laying shortly, then the stores will start going down and we’ll need to keep a close eye on them then. We don’t want them to starve after surviving all winter.

Cheers.

The bees are clustering for winter

Norman — Wed, 23 Dec 2009 19:40:05 +0000

Well, the snows are here and the temperatures have dropped dramatically. The bees are settled in for winter. Hopefully they will make it through to next year (only eight days away!) and be ready to start hunting and gathering pollen, nectar and all the other stuff that they seem to like so much.

We only had them for a brief period at the end of 2009, we got them in late August/Early September but so far they seem to have been fine. Not too much varroa even with ApiGuard in the hive and I’ve just bought them a Christmas present of some Oxalic acid. When I get a decent enough day, I shall apply it and that should hopefully reduce any chance of getting a load of nasty mites in the new year when the brood comes on.

The mouse guard is on and secure, the bees can be seen in the brood box, clustered together for heat and warmth. With the temperatures below zero at the moment, they need all the heat that they can get.

I have 12Kg of baker’s fondant in the garage all ready in case they haven’t got enough stores to make it through. Regular hefting of the hive seems ok at present, but January is when queenie should start laying again, and the stores will take a severe hit when she does.

Here’s to next year.

Cheers.

Get a bloody life!

Norman — Tue, 15 Dec 2009 10:10:28 +0000

Check out this link first.

Speaking as a ginger myself, I have no idea what that woman is complaining about. So what if a card says something funny about gingers (ok, funny is relative, some things I find funny others don’t and vice versa) – it’s called humour and it doesn’t hurt!

I wonder if this woman was simply ‘offended’ because she was after a payout form Tesco for “mental distress” or some other such nonsense. I find that being offended is usually an indicator that you have nothing to say. It’s what the religious fundies are all using as an excuse these days to complain about people saying or doing something that their particular imaginary friend would disapprove of.

The fact that this woman got TV News time and an interview on North Tonight on the BBC is laughable.

Cheers.

We Have Bees!

Norman — Mon, 05 Oct 2009 18:53:27 +0000

Well, it has happened – granted, a few weeks ago – but we have bees. Peter, our mentor was as good as his word and he supplied us with a hive of bees. They were moved by us from a location outside Leeds, West Yorkshire, to our own location. I got stung twice – once on the arm and once on the end of my nose! That certainly makes your eyes water, I can tell you!

Since then, the bees have been transferred from Peter’s hive into our own and they have settled in nicely. After transferring them, Peter, Alison and I spent a happy evening extracting honey from them. Considering the bees had only found Peter’s hive about 4 weeks before we extracted, the fact that we got about 20 pounds (UK) of honey from them was a good haul indeed.

Since then, they have been feeding almost exclusively – or as exclusively as we can tell – on Himalayan Balsam which in the UK is a weed and should be eradicated. This, of course, leaves DEFRA and the Environment Agency with a huge problem. Bees are suffering tremendously at the moment with all sorts of problems and Himalayan Balsam keeps flowering late into the season – it’s still flowering now at the beginning of October – and keeps the bees alive. The other nice thing about HB is that the honey it produces (by way of the bees of course!) is the finest you can taste. What a problem.

As bee keepers (and we have certificates to prove it) Alison and I prefer the HB to stay around for a bit. The bees love it, the flowers are extremely nice – they look like orchids to us – but it’s an invasive weed. Who said life was easy.

Anyway, we are feeding the bees on a sugar syrup at the moment, but not for much longer, as we have robbed them of a lot of honey. The recipe is simple, 2 KG of sugar to one litre of water, heat slowly stirring all the time until dissolved. Allow to cool. If it starts to crystalise add some more hot water and stir again. Easy!

Lets hope we can get these bees through the winter and into next year where they will be able to make as much honey as possible and we will leave them with plenty for themselves. I think it’s better for the bees to feed on their own food rather than the sugar syrup we give them – and every little helps. Bees, at the moment, need lots of help.

The Bees Are Coming!

Norman — Mon, 17 Aug 2009 18:50:47 +0000

Tomorrow night (Tuesday 18th August 2009), Alison and I are off out to a secret location near where we live to colect our very first honey bee swarm. Our Mentor, Peter, has been looking after then for us while we were on holiday, and tomorrow we get to take them home.

We’ve been on the Harrogate & Ripon Beekeepers Association course and are, unfortunately, the last people to get some bees. Some people have simply placed a hive outside and had bees come and live there by themselves, others have purchased bees for themselves (I’m Scottish and Alison is Yorkshire – so that was never going to be an option was it? ) while others have managed to obtain a swarm from their mentors – we just happened to be last!

We are very excited – as you can imagine – and looking forward to the imminent arrival. Hopefully, we can remember all that we were taught by HRBKA and we will make good ‘parents’ to our swarm.

With winter coming soon (well, it is August you know!) we will most likely need to keep them fed throughout the remainder of this year – it is quite late in the season for a swarm. Hopefully this winter will not be as miserable as the last two which caused havoc with the bee populations all over the country.

Wish us luck!

Cheers.

Oracle deadlocks – what happens?

Norman — Wed, 05 Aug 2009 13:30:34 +0000

From time to time an Oracle based application will encounter a deadlock. This happens when two (or more) sessions are holding onto a resource and waiting for another one before it can relinquish the one(s) it holds.

There is quite a lot of misunderstanding about deadlocks and what happens within Oracle to relieve the situation. Hopefully this blog entry will help to sort it all out.

First of all, a brief demonstration.

Background.

CREATE TABLE DEADLOCK (A NUMBER);
INSERT INTO DEADLOCK VALUES (1);
INSERT INTO DEADLOCK VALUES (2);
COMMIT;

Session One.

DELETE FROM DEADLOCK WHERE A=1;

Nothing wrong so far, the row will be deleted.

Session Two.

DELETE FROM DEADLOCK WHERE A=2;

Again, nothing wrong, the row will be deleted.

Session One.

DELETE FROM DEADLOCK WHERE A=2;

This session will hang waiting for Session Two to commit (in which case the delete will fail) or rollback.

Session Two.

DELETE FROM DEADLOCK WHERE A=1;

This session will hang waiting for Session One to commit (in which case the delete will fail) or rollback. In addition, we now have a deadlock condition.

After a few seconds, Oracle will detect the deadlock and pick one of the sessions and ‘rollback’ the statement. This is where we see our first misunderstanding about deadlocks.

Oracle does not kill the session.
Oracle does not kill the transaction.
Oracle only kills the statement.
Oracle does rollback the failing statement, but Oracle does not rollback the entire transaction that the failing statement is part of. (Correction by Mark Bobak.)
PMON (Process Monitor) does not clear out the locks.

It is the responsibility of the session that detects the “ORA-00060 deadlock detected while waiting for resource” error to trap and handle the error by issuing a rollback (or a commit) command. Only once this has been done will the other session be able to continue.

If, like me, you run a test of the above using two separate SQL*Plus sessions (or TOAD sessions, whatever you like) you will find that the session that detects the deadlock will return to a prompt and allow you to enter new commands. The other session remains hung until such time as the other session releases the locks it took out on the rows it had deleted.

This demonstration uses DELETEs but UPDATE will show similar results.

Cheers.

God or Santa?

Norman — Mon, 20 Jul 2009 15:08:23 +0000

I’m not a religious person – I think and check the evidence rather than accepting the word of a misogynistic bloke in a frock.

However when confronted by a religious zealot I find the following conversation goes down rather well! But then, I’m a right b’stard!

Me: Do you believe in Santa Claus?
Religious Person: No, don’t be stupid.
Me: But you believe in god?
RP: Yes, of course.

So what’s the difference? They are both ‘invisible’, both live in ‘far away’ places, both know if you are good or bad, both will reward you for being good or punish you for being bad …

The other thing about RPs in general (ok, the so called Christians) is that when you point out that something in the Bible – all characters and events in this book are fictitious, any resemblance to persons living or dead is entirely coincidental – they always say that I shouldn’t take any notice of that bit.

Excuse me? Is this the word of god or not? Who decides what we pay attention to or otherwise then? If you are picking and choosing which bits of the ‘rule book’ to take note of and which to ignore, then surely you are making it all up as you go along and choosing your own customised version of the bible.

One god? ~~I think not!~~ (Sorry, that link is gone now, there’s an updated one here instead.)

And one last thing, why exactly do they put lightning conductors on church spires then? Have they no faith?

Have Change Control become the new Health & Safety?

Norman — Mon, 20 Jul 2009 14:49:07 +0000

All organisations need Change Control to make sure that changes are applied correctly and so on. However, have Change Control departments become the new “Health & Safety” and gone beyond being useful to being a disruptive and obstructive unit interested only in getting in the way rather than getting work done?

I’m all in favour of change control – it serves a purpose and helps prevent invalid changes being applied to production (or test) systems without them having been checked, tested and applied correctly with full cooperation from the system’s users.

However, I’m seeing a trend develop whereby the cascade or new rules and regulations, redesigned forms, consultation matrices and all the attendant palaver have simply become a joke. Change Control are rapidly hurtling down the road to becoming the new Health & Safety with all the ludicrous diktats that they produce.

These people are left in control of whether a change is applied or not and as to the quality of the code being applied, they care not a jot. However, get a date wrong on a form (filled in in multiple copies of course!) and all hell breaks loose with arrogant emails winging their way around everyone in the entire universe to let them know that you failed to fill in a date correctly.

Finished the task? All went ok? Now fill in a Post Implementation review form or else. Even when there are no actual changes taking place – which they will know from reading the myriad of forms we have sent them prior to getting approval for the ‘change’ – they insist on knowing what was changed and how well it went!

If a change is raised as an emergency, I need to apply it now and right this minute. I do not need or want some numpty who doesn’t know what the change is actually doing or for telling me that it’s not really an emergency because I put a date on to start the work in 5 days time – which is what I have to do in order to get a change approved.

Then, equally, when we agree that perhaps maybe it might actually be an emergency, they then spend the next three days trying to get me to update the work plan to insert the correct end dates and times for the various parts of the change.

My advice is this, change happens, it needs to be controlled, and this is a good thing. However, Change Control departments need to butt out and start doing their job and stop telling the rest of us how to do ours – after all, we don’t go to their office and tell them how to sweep up do we!

I have worked with lots of Change Control people who knew what they were doing, but in recent times they have become few and far between.

PL/SQL expression evaluator

Norman — Thu, 02 Jul 2009 12:43:15 +0000

The following is a pretty nice expression evaluator for Oracle’s PL/SQL language. You pass it a string containing an expression that would return a numeric value when evaluated and it will evaluate the entire expression and return the number.

The passed expression must be in valid Oracle syntax or you will get a NULL instead of a number.

CREATE FUNCTION expression (iExpression IN varchar2)
RETURN number AS

vResult number;

BEGIN
-----------------------------------------------------------
-- Expression evaluator for PL/SQL.
-- Pass in a string containing the expression you want to
-- evaluate using correct Oracle syntax and the result will
-- be returned.
--
-- WARNING: Causes a parse for every expression and will
-- soon fill your cache with similar statements.
-----------------------------------------------------------
-- Norman Dunbar    02 July 2009        Created new function.
-----------------------------------------------------------

execute immediate 'begin :r := ' || iExpression ||
                          ' ; end;' using OUT vResult;

RETURN vResult;

EXCEPTION
WHEN others THEN
RETURN NULL;  -- It all went horribly wrong!
END;
/

Of course, it has its drawbacks, the biggest one being that it will, if called repeatedly with different expressions, fill up your cache with similar statements.

So my challenge is to come up with a similar and equally short PL/SQL routine that will not age potentially useful SQL out of the cache while leaving multiple copies of

begin :r := 2+2; end;

and so on lying around taking up valuable cache space.

Cheers.

If I committed a crime, would the government pay for me?

Norman — Tue, 26 May 2009 12:51:19 +0000

The recent revelations about the inhuman brutality and abuse heaped on innocent children by the so called Sisters Of Mercy and/or the Christian Brothers over 60 odd years in Ireland is an abomination.

Summary of the report

Reactions

But what is worse is the fact that before the report came out damning the Catholic Church for covering up and/or ignoring the abuse (physical, mental and sexual) a couple of things happened.

The Christian Brothers successfully sued in 2004 (the report was 9 years in the making) banning the report from naming names and so no-one can be prosecuted.

The Catholic Church negotiated with the Irish Government with the end result that it is the Irish Tax Payers who are paying the vast majority of the compensation for the crimes committed by the Catholic Church.

Something here doesn’t make sense to me.

Even better, when the Irish citizens discovered this fact and created a row about it (as reported on the BBC news this morning – Tuesday 26th May 2009) the Church simply announced that it would not be renegotiating the deal.

I fail to see how or why the Catholic Church – one of the richest institutions in the world – should be allowed to commit heinous crimes over 60 odd years and then get off by having the people that the crimes were committed against, pay the compensation. Despicable.

My advice to people worldwide who wish to have children is simple:

Don’t. There are far too many people on this planet already and we do not need any more.
If you ignore my advice above, don’t forget to keep your children well away from the Catholics.

Actually, keeping your children well away from any form or religion sounds like excellent advice to me. Live in the real world and not the one favoured by these people who believe in an invisible sky fairy who rules over us all.

Cheers.

Elephant safari

Norman — Thu, 14 May 2009 09:00:13 +0000

Way back in the mists of time, Alison and I spent a wee while in South Africa you can see some of the pictures here. One of the activities we went on was an Elephant Safari.

Well, if this picture is anything to go by, we didn’t have half as exciting a time as these people!

Click picture for the full sized original

Cheers.

Might as well jump on the bandwagon

Norman — Wed, 13 May 2009 13:17:54 +0000

MP’s expense claims seem to be causing all sorts of trouble at the moment.

It’s extremely funny how The Daily Mail suddenly went all quiet on the subject after it was discovered that the Tories had been just as dishonest as Labour. Funny that.

I work for a living and if I get a job in London and have to live there, I can claim a limited amount of my overall expense back. This is fine.

Why is it that MPs, who are doing nothing more than a job – like the rest of us – can claim all sorts of additional expenses over and above what we lowly workers can claim?

I can’t get a load of scatter cushions on expenses for my flat in London, not that I would allow any scatter cushions within the four walls mind you, so why the f*ck should MPs be allowed to? At our expense as well?

When they put themselves up for election they know what the job entails, London, and if they don’t want to have the same overheads of working away from home as the rest of the population, then tough, don’t apply for the bloody job then.

Is it just me that thinks these people have some sort of superiority complex over the rest of us? Being am MP is doing a job, nothing more, nothing less. They get paid for it and incur expenses like we all do. Deal with it in the same way that we have to.

It’s not bloody rocket science after all. (But if it was, then you can bet that the rocket scientists wouldn’t be getting tax payers to fund their scatter cushion fund!)

Cheers.

Bloody cyclists have a new weapon!

Norman — Tue, 12 May 2009 12:45:56 +0000

Driving to work this morning when I hit the Kirkstall Road area where a perfectly good dual carriageway has been split into a bus (and cyclist) lane only and a lane for the rest of us. at the end of this section of the road, there is a set of traffic lights that only affect the non-bus (and cyclist) lane.

The purpose of these lights is to stop the normal people from getting to work on time, while allowing those on a bus the ability to get about 20 yards further up the road than they would normally if there were no lights there. Go figure, this is Leeds after all.

The lights are operated – in other words, they turn red – when a bus or coach passes over a sensor in the road about 300 yards back from the lights themselves. This allows the bus to trigger said sensor and travel at breakneck speed – as they do – towards the lights which will have turned red by the time it gets there.

Unfortunately, this morning, all the bloody cyclists were triggering the sensor as well. So every bloody cyclist passing over the section of road where the sensor was buried, turned the lights red. We all have to stop while no buses are coming and the cyclists are so far away from the lights that they will be green by the time they get there.

What a crock of shit!

What’s wrong with this file system?

Norman — Mon, 11 May 2009 10:02:24 +0000

On the QL we have traditionally had a 36 character limit on file names after the initial (5 character) device name. Something like this:

flp1_source_code_myfile_c

In the above, flp1_ is the device name and is not included in the 36 character limit. There can be a network device tagged on the front as well, and this too is not included in the filename limit.

Technically, in the above, source and code are both directories (the default directory separator is the underscore character) while myfile_c is the filename and extension.

In the original days of the QL, all we had was a pair of built in microdrives and so the file name limit wasn’t too much of a hassle because we only had about 100KB (yes, KB) to play with and there were no actual directories as such.

Then we moved on to floppies and even a 40 MB (yes, MB) hard drive – and suddenly, this limit began to look rather silly, especially as we now had proper directories.

The problem, as I see it is simple. Inside the directory entry for a filename, a space of 36 characters has been reserved. Unfortunately, given the above filename as an example, the following will happen:

The root directory on the floppy disc will contain an entry for a directory named source.
The source directory will have an entry for a directory named source_code.
The source_code directory will have an entry for a file named source_code_myfile_c.

You can see the problem, the entire directory structure, less the device name, is replicated all the way down the tree. Surely it makes sense, even with a 36 character limit, to only have the appropriate parts of the full path in each directory. For example:

The root directory on the floppy disc will contain an entry for a directory named source.
The source directory will have an entry for a directory named code.
The code directory will have an entry for a file named myfile_c.

With this system, we would be able to have almost unlimited depth to our disc structures – possibly not wise – and each part of a file’s full path would be limited to 36 characters, not the entire path itself.

Well, I think it makes sense anyway.

Cheers.

A different way to spend a Saturday …

Norman — Mon, 11 May 2009 09:40:45 +0000

Alison and I went to Harrogate on Saturday and spent the day building a couple of beehives. Alison hasn’t done any form of wood workery before and she did a really good job of putting her hive together, although her hammering technique was a tad amusing, but then again, as she’s never had to use a hammer before why should she know?

Me? Well, I’m a professional (mechanic and DBA) and I managed to get a bit of mine glued and nailed into the wrong location and, I think, upside down as well! Duh! A quick bash with a hammer and chisel and it was retrieved ready to be fixed onto the right place.

At present the hives are store in the garage and are exuding a very pleasant “woody” smell. Hopefully soon we will get them into the garden and full of bees.

For details of bee keeping in the West Yorkshire area, check out http://www.hrbka.org.uk/ which is, if I do say so myself, a rather fine website.

We also prepared for the arrival of our new workers by purchasing a smoker, a hive tool, some frames and foundation (for the brood box) from Claro Bees and ordering a couple of rather natty jackets with a built in veil. Both in bridal white as well so we shall look very dashing indeed!

Claro Bees is an intersting shop. It looks like a portakabin!

Cheers.

I hate those damned koi carp!

Norman — Wed, 22 Apr 2009 19:44:11 +0000

As mentioned previously on this blog, we have had a large number of breeding frogs in our pond.Well the tadpoles have hatched and are growing well. At least, the ones that manage to stay well hidden are.

Those damned Koi are eating machines and there is wholesale slaughter going on in the pond at the moment. The volume of tadpoles is dwindling quite rapidly as those fat, greedy bastards (have I mentioned that I don’t like them?) gorge themselves on anything that moves.

Why can’t they eat the cats that we get in the garden I wonder?

I’d love to don my scuba gear and go spear fishing in the pond. Only kidding! You can’t use scuba gear when spear fishing in the UK – so I’ll have to use my snorkel instead!

At least I managed to save afew lumps of spawn and move them to the little ponds in the back garden for safety. No fish in there – just how I like it. Those little ones are doing well so far.

Cheers.

Why is it that the people with the worst taste in music, play it loudest?

Norman — Wed, 22 Apr 2009 19:38:11 +0000

Walking through Leeds today, traffic doing its thing as usual. Then, in the distance, it could be heard…..….. thump thump thump getting louder and louder and louder. Soon, it appeared, yet another bloody moron in a beat up old Corsa (or similar) with a dustbin exhaust – take note, that changes the engine’s breathing and will stuff your engine a lot quicker than you think. Intake and exhaust are balanced for best results by professional engineers, and you are not one of those.

The noise, for it could hardly be described as music, is pumping out with maximum base applied and the volume turned up as loud as it will go (all the wat to 11 for Spinal Tap fans!).

Now, I love my music, but I listen to it, not everyone else on the bloody planet! I am smart enough to realise that not everyone has my taste in music, so I don’t force everyone to listen to it. Why can’t these driving morons have similar consideration for the rest of us?

There is an additional problems too, not just the noise. The volume is so high that whatever music was intended to be played has no chance thanks to the total harmonic distortion of about 100% introduced by the crap quality speakers! You can usually hear them trying desperately to escape from their mountings.

The ‘C’ is silent – as in Rap Music.

Political correctness my arse!

Norman — Tue, 21 Apr 2009 12:02:15 +0000

As I came into work this morning (21st April 2009) I was listening to the man from the Driving Test ‘agency’ on Radio 4.

The person was asked about the proposals for adding a new section to the test where the examiner gives the trainee driver a set of instructions similar to directions obtained from a passer by, and whether a trainee would be failed for getting the instructions wrong.

The man from the agency replied , with a straight face, that “we do not fail people. We have people who pass and those who don’t pass”. The Radio 4 interviewer practically wet himself laughing.

Me? I was screaming at the radio again! What the f*ck are we doing to this country with all this crap about political correctness and not hurting people’s feelings? It is all lies I tell you, damned lies.

There are failures in exams, driving tests and so on, no matter what you call it, they are failures and it is a fact of life. Stop trying to kid on that people don’t fail, they effing well do – I should know, I failed my maths Higher (A Scottish Educational exam) where you get a grade between A and F, I got nothing, not a mention, zilch, bugger all in fact.

I failed it so miserably indeed.

Am I bitter? No.

Has it affected my life since them? Not yet.

Do I care that I failed, well yes I do, but I know for a fact that I did fail, I didn’t not pass!

Cheers.

God and the Italian earthquake

Norman — Thu, 09 Apr 2009 12:34:27 +0000

I was shouting at a nun on TV the other night. The silly woman had [thankfully] survived the Italian earthquake and in retelling her story she said words along the lines of “I woke up when the quake started and the walls began to fall down. I got up and rushed outside. I only escaped because of god.”

Does the woman not realise that it was her ‘god’ who was responsible for the damned earthquake in the first place? Or does that not count? Funny how ‘god’ is only ever mentioned for the good stuff.

Why do they put lightning conductors on church spires then?

The frogs are gone …

Norman — Thu, 19 Mar 2009 14:12:34 +0000

Well, the frogs have almost gone. After a mass orgy of croaking all night and day (do they ever sleep?) and sex all day and night (lucky b*ggers!) they have started leaving. The pond is returning to the domain of the Koi carp (which I hate almost as much as I hate cats) and is full of spawn. This morning there were only about a dozen frogs left, a couple still paired up, but most of the rest had gone. Where to I wonder?

No doubt that damned cats will be having a field day killing and maiming the poor b*ggers as they wander back to whence they came. Wish I could do the same for the cats – send them back to wherever they came from I mean. (Ok, I admit it, in a box if necessary!)

Funnily enough, this year we have no frogs in the ponds at the back of the house, only in the one at the front. Last year we had lots at the back as well – but that was where the cats used to hang out – until I put the cat zapper in. I just wonder if those damned murdering b’stards managed to kill off all the frogs at the rear?

Anyway, we seem to have a leak in the smallest pond at the rear, so I’ll have to get patching soon.

Cheers.

Live badger spotted in Yorkshire!

Norman — Thu, 12 Mar 2009 12:27:13 +0000

Coming home from our bee keeping course on Tuesday night (10th March 2009) Alison and I spotted a ghostly shape on the grass verge in the dark. Turns out to have been a live badger. We stopped (yes, it was safe to stop!) and it wandered away a few yards, paused and watched us watching it. We remained thus for about 10 seconds until it lost interest in us and wandered off into a field.

At least it wandered away from the road!

I’ve only ever seen wild badgers as corpses until now! Usually at the side of the road where so many end up. I have to say that they are pretty hard to see in the gloom. Must be something to do with their colouring and us humans being pretty blind at night as well!

Foxes are even more difficult to see at night – unfortunately, it’s usually too late when you do spot them!

Cheers.

The frogs are spawning again

Norman — Mon, 09 Mar 2009 10:49:18 +0000

Just like last year, the mass orgy of frog sex is taking place on Alison’s birthday! We have more mating frogs that you can shake a stick at in the pond right now. Most are spawning – as they do – some already have. It’s interesting to see how the males lose interest in the females once sex has been had!

One problem I have this year, lots of the frogs are getting up the pump outlet pipe and into the filtration system. I’ve extracted about 20 pairs of frogs from the filter bed this weekend already. I don’t want them going up there and spawning as they will clog up the works and, most likely, die as they cannot get out and there’s nothing to eat in there either.

All I need now is some method of stopping them getting up the pipe and, much more importantly, some method of preventing those damned Koi from eating all the tadpoles. I have suggested a harpoon but I have been over-ruled by Alison on the matter as she loves her fish.

Cheers.

Oracle diagnostic events

Norman — Thu, 05 Mar 2009 09:41:32 +0000

Tanel Poder has started what looks to be an excellent series of posts on his blog on the subject of Oracle diagnostic events (eg 10046 traces) at this location.

Another cycling moron

Norman — Mon, 23 Feb 2009 20:47:33 +0000

I ranted previously about cyclists. Tonight, coming home from work, I saw the biggest moron on two wheels yet.So there we were at a junction. I was at the head of the queue on a dual carriageway. There was traffic crossing from our left and a cyclist arrived up the inside of a large van and cycled to the head of the queue and pulled in front of the van!

The cyclist had no protective clothing or head gear at all. She, for it was a woman, was dressed in her sunday best by the look of things! She was now at the front of the queue, however, her position on the road put her slap bang in the middle of the other side of the dual carriageway and right in the path of the oncoming vehicles.

This woman was completely unaware of the carnage heading her way as she started to pull out into the road without looking in any direction what so ever! She was missed – very closely – by at least 4 cars. None of them much further away from destroying her than 6 inches. I kid you not.

Once she had managed to avoid all the oncoming traffic, she wobbled directly into the path of the van she had just queue jumped. Still not looking and still, miraculously, breathing and unhurt. As the van took avoiding action to avoid this lunatic, the driver tooted his horn. His reward for not killing the cyclist?

She gave him the finger.

I suspect this woman is not long for the world. Some people should not be allowed to breed!

Cheers.

The frogs are coming!!!

Norman — Tue, 17 Feb 2009 08:07:39 +0000

Last night Alison found the first frog of the year in our driveway. The snow that has been around for a couple of weeks now had vanished as the temperature raised itself to a heady 6 degrees C and out she came, (the frog, not Alison!) heading for the big pond.

Last year, on March the 9th (Alison’s birthday), I counted over one hundred froggy heads in our pond by the front door. It was very pleasant sitting in our lounge listening to them sing the evening away. About a week or so later, they were gone but their spawn filled the pond. When it had all hatched, the water was black with tadpoles. Nice!

Unfortunately, when we bought the house, the pond we inherited was full of fish too. Some Koi Carp and some yellow things and a couple of black versions of the yellow ones – can you tell I know what I’m talking about – and I’m sad to say that they completely ‘dysoned’ the tadpoles up leaving nothing at all. I never saw a single young frog from the big pond last year.

We did get a few froglets from the smaller ponds round the back of the house as there are no fish there. Unfortunately, all the neighbourhood cats used to play around there and hunt the frogs, squirrels and birds. Murdering bastards!

“They are only doing what comes naturally” cat owners are fond of saying and this is true, but if I had a python and it did what they do naturally, to their cats, I’m sure they would change their tune! Keep your effing cats out of my garden! I’m pretty sure that if I took all the cat shit back to the owners they wouldn’t be too enamoured either – so why do I have to put up with it?

I hate cats by the way, can you tell?

I’m happy for you to own a cat but keep the effing things inside where they can do no harm otherwise, don’t be surprised if they don’t come home one night – if only I could set up a cat killing laser defence system around my garden life would be grand! For the moment, I have to be satisfied with the RSPB approved cat deterrent device which emits ultrasonic sound that cats, dogs and foxes find uncomfortable. It seems to work!

I’m also not all that fond of the fish either, but I do like frogs. Unfortunately, Alison likes the fish and indeed, the cats, so a mass slaughter of these killer alien creatures is somewhat forbidden at the moment I’m afraid to say. She won’t let me buy a crossbow from Amazon either. Is there no justice in the world?

With a bit of luck, I’m hoping to help boost the frog population this year by grabbing a load of spawn from the big pond and using it to fill the smaller ponds at the back of the house. They should be safer there this year as the cats now populate the front garden instead.

I’d much rather I just slaughter the fish (foul creatures that they are) and the cats too. Still, there’s time!

Cheers.

It must be efficient, I’m using bind variables!

Norman — Wed, 04 Feb 2009 13:17:06 +0000

For many years, various big guns – and a lot of smaller ones – in the Oracle world have been advocating, nay demanding, that we [almost] always use bind variables in our SQL code. The reason is simple, it’s shareable, efficient, reduces parsing and allows the application to scale up to more and more users.

Over the years I have spent fixing and tuning Oracle databases, I have noticed a trend of developers moving away from hard coding everything to using binds more and more in their code. How refreshing, the performance must also be improving, mustn’t it?

Not quite!

The problem is becoming apparent that a lot of developers don’t know anything/much about Oracle and also don’t know anything/much about their own chosen language either.

I’m pleasantly surprised when I see code like the following in the cache on my databases :

SELECT stuff FROM some_table 
WHERE some_column = :a_bind_variable;

It’s a start, but it is still broken. How can this be?

While the developer is indeed using binds, s/he has not considered how that statement is created within the application. As there are many application coding languages, I shall resort to my own pseudo-code to demonstrate the problem.

Function GetEmpForID(int empId, Connection conn)
begin
 Statement Stmt(conn, "Select stuff from some_table 
                       where some_column = :some_value");
    Stmt.Parse();
    Stmt.Bind(":some_value", empId);
    ResultSet Rslt =  Stmt.Execute();
    //Process Results here
end

Looks great doesn’t it? Well, no, it doesn’t. The major problem is that fact that the statement will be parsed each and every time that this function is called. Within a function (in most languages) the statement is created on the stack and on exit from the function, deleted.

In reality, we don’t get much in the way of efficency improvements because we still have a 1:1 parse:execute ratio, not what we want at all. What we need to do is make the statement external to the function (global or whatever the language supports) similar to the following.

Connection conn;
Statement StmtEmpForId(conn, "Select stuff from some_table 
                       where some_column = :some_value");

Function GetEmpForID(int empId)
begin
    external Statement StmtEmpForId;
    if not StmtEmpForId.Parsed() then StmtEmpForId.Parse();
    StmtEmpForId.Bind(":some_value", empId);
    ResultSet Rslt =  StmtEmpForId.Execute();
    //Process Results here
end

Now we are getting somewhere! The statement is external to the function itself, so exists even when the function has exited. If this is the first time that we call the function, the statement will be found to be unparsed, so we can parse it – this avoids the application parsing statements at startup and possibly parsing statements that never get used.

Once parsed, and on every subsequent call of the function, all we have to do is bind the variable and execute the statement before processing the results.

As long as the statement (and connection) remain in scope (hence being globals) then we only parse the statements once regardless of the number of times that we execute it. So, if we execute our one statement a million times, we have only carried out one single [hard] parse.

A good workman knows how his tools work and how to use them.

In C++ you would, I suppose, create an appConnection object with members for the Oracle (OCCI?) Connection object and each of the Statements you wish to use in the application, after all, the statments should remain in the same scope as the connection shouldn’t they?

In Java, well, I have no idea, all the Java stuff I’ve looked at seems to be reinventing the wheel over and over again – who knows what those guys get up to!

Cheers.

ROWIDs are fun

Norman — Mon, 26 Jan 2009 13:20:23 +0000

In a previous posting here on the subject of Lazy Developer Syndrome, I showed a small fragment of code where I SELECTed the ROWID in addition to all the other data I wanted, then UPDATEd the same row using the ROWID I had stored rather then using the Primary Key index that I used to SELECT the row in the first place. Why did I do this?

Selecting the ROWID is always a good idea if you intend to UPDATE or indeed, DELETE, it afterwards. The ROWID is after all the fastest way to get to a row in your table. It’s how the indexes get you to the data after all.

If you have a primary key constraint and the index supporting it has a height of 2, then to get from your supplied primary key value to the data requires 3 single block accesses – the index root and the correct leaf block. From the leaf block we get the ROWID for the requested primary key value, and a further single block read takes place to fetch back the block containing the required row.

Some of these blocks may be in the cache but then again, they might all not be – so 3 physical reads would be required as the worst case and 3 logical reads as a best case.

An UPDATE specifying the primary key value again will also have to perform 3 more single block reads, hopefully from the cache. However, an UPDATE using the ROWID already fetched will only have to perform one block read, again hopefully from the cache, and that block will be the actual data block to be amended. We don’t need to use the index and we save 2 single block reads. Good eh?

A DELETE using a ROWID or a primary key value will also take differing numbers of block reads. If you delete using the ROWID, the index has to be read and updated by removing the entry for the row in question and if you DELETE using the Primary Key value, the index will be read again (in consistent mode) and the data block will be read in current mode in order to make the DELETE work.

My own testing, so far, has shown that the difference is a single block read saved when using the ROWID rather than the Primary key value however, I have not yet determined if that figure will change if the height of the index itself changes.

Update by Primary Key.

SQL> update indextest 
     set name = name 
     where id = 86057;
1 row updated.

Execution Plan
----------------------------------------------------------
 0 UPDATE STATEMENT Optimizer=ALL_ROWS 
    (Cost=2 Card=1 Bytes=79)
 1 0 UPDATE OF 'INDEXTEST'
 2 1  INDEX (UNIQUE SCAN) OF 'INDEXTEST_PK' (INDEX (UNIQUE)) 
       (Cost=1 Card=1 Bytes=79)

Statistics
-----------------
1 db block gets
2 consistent gets
0 physical reads
1 rows processed

Update by ROWID.

SQL> update indextest 
     set name=name 
     where rowid = 'AAAXktAAEAAAET8ABI';
1 row updated.

Execution Plan
----------------------------------------------------------
 0 UPDATE STATEMENT Optimizer=ALL_ROWS 
   (Cost=1 Card=1 Bytes=78)
 1 0 UPDATE OF 'INDEXTEST'
 2 1 TABLE ACCESS (BY USER ROWID) OF 'INDEXTEST' (TABLE) 
       (Cost=1 Card=1 Bytes=78)

Statistics
-----------------
1 db block gets
0 consistent gets
0 physical reads
1 rows processed

Delete by Primary Key.

SQL> delete from indextest 
     where id = 86057;
1 row deleted.

Execution Plan
----------------------------------------------------------
 0 DELETE STATEMENT Optimizer=ALL_ROWS 
    (Cost=1 Card=1 Bytes=13)
 1 0  DELETE OF 'INDEXTEST'
 2 1   INDEX (UNIQUE SCAN) OF 'INDEXTEST_PK' (INDEX (UNIQUE)) 
        (Cost=1 Card=1 Bytes=13)

Statistics
-----------------
5 db block gets
2 consistent gets
0 physical reads
1 rows processed

SQL> rollback;
Rollback complete.

Delete by ROWID.

SQL> delete from indextest 
     where rowid = 'AAAXktAAEAAAET8ABI';
1 row deleted.

Execution Plan
----------------------------------------------------------
 0 DELETE STATEMENT Optimizer=ALL_ROWS 
   (Cost=1 Card=1 Bytes=25)
 1 0  DELETE OF 'INDEXTEST'
 2 1   TABLE ACCESS (BY USER ROWID) OF 'INDEXTEST' (TABLE) 
        (Cost=1 Card=1 Bytes=25)

Statistics
-----------------
5 db block gets
1 consistent gets
0 physical reads
1 rows processed

SQL> rollback;
Rollback complete.

SQL> exit

So, when creating your application, the idea is to build in performance from the start (ie, you do not add it in after the fact!) so when reading data that will be updated or deleted in a subsequent operation, always grab hold of the ROWID and use that to make your DELETE or UPDATE as efficient as you can.

By the way, never (can we say ‘never’ in these politically correct times?) store the location of a row in another table as a ROWID – if you do, and you ever export the table and reimport it, the ROWIDs you carefully saved will no longer point to the row that you thought they did!

Did you never wonder why indexes are stored in the export file as a CREATE INDEX statement, and not as the pure index data itself?

Cheers.

Lazy developer syndrome

Norman — Thu, 22 Jan 2009 15:23:24 +0000

To those who don’t know me, I’m an Oracle DBA and I also can develop as well. I detest having to work on applications which have the following construct in the code:

SELECT 
FROM TABLE
WHERE 
FOR UPDATE;

In most cases, the above is a sign of developer laziness. They cannot be bothered to write correct code to handle a situation where a row that the user has been working on (unlocked) has been amended by another user in the meantime.

In Oracle you should lock late and lock short (as Dave Ensor says). Only lock a row as you are about to update it. This is optimistic locking.

Do not lock it when you start working on it and keep it locked until you finish the update. This is pessimistic locking. When you write an application that does this, you end up with rows locked out all over the place as people get half way through amending some details and then have to answer the phone, or pop outside for a ciggie, or go to the loo.

Pessimistic locking – just say NO!

It isn’t difficult, for example, to add a column to the table to hold an update counter (or similar) and to have a trigger update that counter by one each time an UPDATE is performed. The developer would read in the current value along with the data he wishes to display for amendment and the ROWID for the row in question. Once the user has made the changes and clicked OK (or whatever) what’s wrong with running the following statement :

UPDATE
SET =
WHERE ROWID=:stored_rowid
AND upd_counter=:stored_upd_counter;
If it works, job done, slip in a COMMIT while no-one is looking, that will release the locks and will save the data permanently to the table with the changes made and the trigger’s change to the update counter carried out as well.
If, on the other hand, it returns no rows updated, we know some other user got there before us and changed the value in the update counter column. Now, we can handle that by displaying a message to the user and perhaps switching to a new screen showing the user’s own changes and the other user’s changes and then request that our user chooses to either force through his own changes, accept the other users changes aborting his own or to redisplay the previous screen with the new data loaded for further amendments.
It’s much more friendly for all users of the application but it takes a little more thought on the developer’s side of things. It’s easier for a lazy developer to simply choose not to think and blindly carry on with his SELECT … FOR UPDATE nonsense.
Cheers.

So transport is killing the planet is it?
Norman — Tue, 20 Jan 2009 16:51:04 +0000
I suspect not.
http://www.newscientist.com/article/mg20026873.100-how-kangaroo-burgers-could-save-the-planet.html
The gist of the above article is this, worldwide, livestock creates more greenhouse gasses than all forms of transport combined. Eating meat is bad for the planet – can you hear all those veggies crowing now? Never mind, Kangaroos taste good and can help save the planet. Check out the link above for the full story.
Cheers.

Cyclists – I hate them!
Norman — Tue, 20 Jan 2009 16:40:26 +0000
I hate cyclists. They are so full of themselves! They sit there, two or three abreast on narrow roads, dawdling along having a nice chat, completely oblivious to the queue of traffic behind them. Give them a happy toot (ie, not a nasty long drawn out blast) on the horn to let them know you are there and you get a load of abuse. Nice & friendly!
I was once run over by a cyclist. I was on a pedestrian crossing, with the green man telling me I could cross. The cyclist [must have] overtaken a queue of stationary traffic to get me, but he managed it. He was apparently watching some scantily clad women on the other side of the road, his ipod (other personal stereos are available ) was turned up as loud as it would go and he hit me.
When he saw what had happened, his excuse was that “the lights were green!” which was highly amusing to the taxi driver at the front of the queue (he said as much out of his window while calling the cyclist a few choice names too) as he had overtaken the taxi in order to get me!
I have also been narrowly missed on numerous occasions when the lights were red (for traffic) by cyclists (usually clad in those tight shiny suits) who apparently do not have to stop at the red lights. When I comment as they pass, that “the lights are red” – they all seem to have serious doubts about the marital status of my parents at the time of my conception! Hang on, I’m the one with right of way so why am I the ‘bastard’?
Don’t bother to mention one way streets either, they are apparently not applicable to cyclists.
Ok, I’m not tarring all cyclists with the same brush, there are many who do have consideration for other road users and indeed the laws of the road and Highway Code too. But so far, not many!
They pay no road tax, no insurance and demand that we give up half of our roads to enable them to have their own dedicated cycle path. Excuse me? In my opinion, cyclists shouldn’t be allowed on the road until they have taken a test and passed it (I at least passed a cycling proficiency test before I was allowed out on the road by my parents) and you should pay at least third party insurance as well. Then you can consider yourselves ‘allowed’ on the public roads (ok, I get the irony of that last bit!) .
If I am driving my car along the road and a cyclist ignores the lights and gets hit by me, I have no come back on him for the damage to my car, caused by him/her and if s/he is dead, how do I claim back from the corpse? Try it and all you will get is a tabloid style attack from the various red tops about how a callous car driver killed an innocent cyclist and then billed his parents for the damage. Best you get insurance then!
To all you human, considerate, and law abiding cyclists out there – well done and thank you. You have my sincere sympathies at the treatment you get from other road users purely because of the insane and selfish behaviour of a number of your peers.
To the rest of you, and you know who you are, get stuffed!
By the way, two bits of black tarmac in the pub having a chat. A red bit of tramac walked in. The bar went quiet. One of the black bits turned to the other and said “do not say anything to him, he will kill you!”. The other replied “Why?”. “Because he’s a cycle-path!”
Are cyclists really friendly to the environment? I don’t think so myself. All that puffing and panting as they cycle slowly up hills must surely be contributing extra CO₂ to the atmosphere? (That wasn’t a serious question by the way!)
Cheers.

Hello world!
Blogman — Tue, 20 Jan 2009 13:19:14 +0000
Welcome to my blog.
Don’t expect me to be sensible or politically correct here, it’s my space and I’ll say what I like! So there!
Of course, anything I do say will be able to be commented on by you, my faithful reader (singular!), assuming that you care of course.
Ok, seriously, I’ll be ranting and raving about all sorts of stuff here. I’ll try to categorise them so you can sift the interesting from the less interesting. Who knows, it might be amusing.