There are many online resources to explain pipelined functions so I will not do this here.

Here is the code:

-- create object
create type space_usage_object as object 
(
   Segment_owner varchar2(30),
   segment_name varchar2(30),
   segment_type varchar2(30),
   partition_name varchar2(30),
   blocks_unformatted number,
   bytes_unformatted number,
   
   blocks_0_25 number,
   bytes_0_25 number,   

   blocks_25_50 number,
   bytes_25_50 number,   
   
   blocks_50_75 number,
   bytes_50_75 number,   
   
   blocks_75_100 number,
   bytes_75_100 number,            
   
   blocks_full number,
   bytes_full number
);


-- create type
CREATE TYPE space_usage_type AS TABLE OF space_usage_object;   

-- pipelined function
create or replace FUNCTION SPACE_USAGE (
  in_segment_owner        IN  VARCHAR2 ,
  in_segment_name  IN  VARCHAR2 ,
  in_segment_type  IN  VARCHAR2 ,
  in_partition_name       IN  VARCHAR2 := null)
  RETURN space_usage_type PIPELINED AS

 v_unformatted_blocks number;
 v_unformatted_bytes number;
 v_fs1_blocks number;
 v_fs1_bytes number;
 v_fs2_blocks number;
 v_fs2_bytes number;
 v_fs3_blocks number;
 v_fs3_bytes number;
 v_fs4_blocks number;
 v_fs4_bytes number;
 v_full_blocks number;
 v_full_bytes number;
 
 
begin


  dbms_space.space_usage(segment_owner => in_segment_owner,
                        segment_name => in_segment_name,
                        segment_type => in_segment_type,  
                        unformatted_blocks => v_unformatted_blocks, 
                        unformatted_bytes => v_unformatted_bytes,
                        fs1_blocks => v_fs1_blocks, 
                        fs1_bytes => v_fs1_bytes,
                        fs2_blocks => v_fs2_blocks,
                        fs2_bytes => v_fs2_bytes,
                        fs3_blocks => v_fs3_blocks,
                        fs3_bytes => v_fs3_bytes,
                        fs4_blocks => v_fs4_blocks, 
                        fs4_bytes => v_fs4_bytes,
                        full_blocks => v_full_blocks, 
                        full_bytes => v_full_bytes,
                        partition_name =>  in_partition_name);
                        
  pipe row (space_usage_object (in_segment_owner, in_segment_name, in_segment_type, in_partition_name,
                              v_unformatted_blocks , v_unformatted_bytes, 
 v_fs1_blocks, v_fs1_bytes, v_fs2_blocks,  v_fs2_bytes,  v_fs3_blocks, 
 v_fs3_bytes, v_fs4_blocks , v_fs4_bytes , v_full_blocks , v_full_bytes ));

  RETURN ;
END;


-- selecting 
select * from table(space_usage ('OWNER', 'SEGMENT_NAME', 'SEGMENT_TYPE'));

Hazem Ameen
Senior Oracle DBA

Our test cases were run in our development environment which is 10.2.0.5 RAC sitting on HP Itanium.

The target was to delete a few million rows from 2 tables which were related 1 to many with an enabled foreign key  “on delete cascade”.

The initial plan was simple:

1)      Delete from child table first so we can eliminate lookups from parent to child.

2)      Delete from parent .

3)      Use a nice parallel hint for both steps.

We tested our plan with about 35,000 rows.

First step went fine, but second step the query went for a long time, when we checked, we found that query is waiting for TX Lock even though we executed the query in the same session.

Why wait for TX Lock in the same session?

This is because the first query was using a parallel hint which created parallel sessions.  These parallel sessions locked the rows in the child table. You couldn’t move forward unless we issue a commit.

You can also produce the same behavior in a single instance environment without using any parallelism by doing the following simple steps:

1)      Delete child row

2)      Create a new session and then delete the parent row.

The deletion of the parent row will hang until you commit in the child row.
 

Foreign Keys Performance

The following test cases were executed for about 35,000 rows. Every time we delete, we copy the data back.

We deleted child data first, committed and then deleted from parent table, it took 75 seconds

We deleted directly from parent table, it took 75 seconds (apparently there is no gain from deleting from child table first then deleting from parent table, bummer!)

We disabled foreign keys, delete from child, then delete from parent, total time was 12 seconds only, 6 times as fast.

Conclusion

Foreign Keys are excellent for ERD diagrams and remind everybody which tables are related, but physically disable them and you should live happily every after.
 
Hazem Ameen
Senior Oracle DBA

Timestamp data type adds two new parts to the regular “Date” data type:

1. Fraction of a second in microseconds ( 6 digits)
2. Time Zone (shown only on selected functions like systimestamp and current_timestamp and shown on data types such as “Timestamp with time zone”)

Break down of Timestamp: 2/9/2011 10:39:36.802604 AM +03:00
“2/9/2011 10:39:36″ is exactly like older Date date type.
“802604″ is a fraction of seconds only shows only in timestamp data types.
“+03:00″ is Riyadh Time Zone.

Why do I need a fraction of a second in my date, isn’t seconds enough?

For most applications, up to second precision is enough, but some applications do require microseconds.

Applications like trading, biding such as eBay, and some physics applications were the operations start and finish in microseconds can benefit from timestamp.

What is Time Zone?

The actual definition you find it here http://en.wikipedia.org/wiki/Time_zone

Time Zone are like EST (Eastern Standard Time) or CST (Central Standard Time), etc… is determined by how far you are from Greenwich Line.

Here in Riyadh we are UTC + 3.00

Do I need Time Zone?

Applications that gets deployed across multiple time zones will benefit from this feature greatly. Worldwide internet applications with one central database  or replicated worldwide databases  are some of application that utilize Time Zone. Prior to this Time Zone feature, we had to create a Date column that hold  database server date/time with database server time zone and an additional Date column that holds date/time with client’s time zone. Lots of manual conversion (not to forget Day Light Saving Time!). A time zone mess in short.

Here is a table that shows Timestamp data type compared to the older Date data type.

 
How can I know my database time zone?
SELECT dbtimezone FROM DUAL;

 
How to know client’s time zone?
SELECT sessiontimezone FROM DUAL;

 
How to change database time zone?
ALTER DATABASE SET TIME_ZONE = ‘+03:00′;

Then restart the database.
 
How to change client time zone?
ALTER DATABASE SET TIME_ZONE=’+03:00′;

OR by using a named region
ALTER DATABASE SET TIME_ZONE=’Asia/Riyadh’;
 

This later “alter” will change current_timestamp  output to :
13-FEB-11 11.54.19.985565000 AM ASIA/RIYADH

OR by using environment variables

export ORA_SDTZ=’+05:00′

 
How to initially set up your database time zone?
CREATE DATABASE testdb
. . .
. . .
SET TIME_ZONE=’+03:00′;

 
Where can I find a list of all time zones?
Look up this view: V$TIMEZONE_NAMES

Hope this helps.
 
 
Hazem Ameen
Senior Oracle DBA

Test Details:
Table: 2 GB approx, no LOBs, source and target tables are not partitioned
Oracle version: 10.2.0.4 installed on HP Itanium with 4 CPUs
Instance: Single instance ( no RAC), database in noarchive mode
File System: UFS

Here are the test results:

 
Hazem Ameen
Senior Oracle DBA

Before detailing our scrambling solution, here is a common question: What is the difference between encryption and scrambling.

Here are some of these differences:

How to Scramble Data in Oracle

The best way to perform scrambling is through translate built-in function.

Here is an example:

SELECT 'Hazem Ameen' as before_scrambling , TRANSLATE(lower('Hazem Ameen'),
'.@,0123456789abcdefghijklmnopqrstuvxyzابتثجحخدذرزسشصضطظعغفقكلمنوهي', '.@,4214563875qwertyuiop[kjhbvabcdefxzgباتنمكضصثقفغعهخحجدظزوةىرذشسؤ') after_scrambling
FROM dual;

It is really up to you how you want to apply it in your environment. At the end would have to execute a statement like this on every column that requires scrambling:

Here are some tips on how to apply this in your environment:

1. If you have too many columns or the process of scrambling repeats often, insert table name along with columns to be scrambled in a table, then develop a PL/SQL procedure to read this table and execute the update statement dynamically.
2. Updating large amount of data via a single update statement will take a very long time; instead you would open a cursor and loop through the data.
3. Be aware that triggers on the table will slow down this procedure dramatically. If you can disable them first before running this procedure.
4. This scrambling process doesn’t scramble dates or LOBS.

The original code to copy data from a remote view consists of 3 steps:

1. Copy over employee numbers (one column only) from remote view to local temp table. This step takes few seconds only.
2. Use employee numbers copied locally to do a lookup serially (lookup row by row) from remote view. This step takes about 4 hours to copy 20,000 rows. The reason it takes that long is one column from the view is an actual function. Most of the time is spent executing this function and for so many reasons (not all are technical) we can’t get to function to tune it.
3. Commit one time at the end.

Here how the original code looks like:

– Step 1:
– truncate local temp table so we can copy employee numbers
– copy over employees numbers to local temp table from the remote view

execute immediate 'truncate table temp_emp_no';
execute immediate 'truncate table employees';

insert into temp_emp_no select distinct EMPLOYEE_NUMBER from remote_employee_view@remote_link;
commit;

– Step 2:
– Loop through copied over employee numbers and do a remote lookup 1 row at a time

cursor c1 is
   select emp_no from temp_emp_no;

for i in c1 loop
   exit when c1%notfound;
   insert into employees
   select EMPLOYEE_NUMBER, first_name, last_name, RATE, PERIOD, total_rate
     from remote_employee_view@remote_link;
   where EMPLOYEE_NUMBER = i.emp_no;
end loop;

commit;

To simulate multithreading we need to accomplish 2 steps:

1. Break the job in multiple pieces (threads).
2. Schedule every thread to run concurrently.

Remember first step which is coping employee numbers only from remote view to local table. We hash-partitioned this local table into 4 partitions. Each of these partitions will translate into a thread as you will see.

CREATE TABLE vb_emp_no
   (emp_no  VARCHAR2(30))
    PARTITION BY HASH (EMP_NO)
     PARTITIONS 4
/

CREATE procedure refresh_employee_data_part (p_name in varchar2 )
  authid definer
is
  TYPE EmpCurTyp  IS REF CURSOR;
  v_emp_cursor    EmpCurTyp;
  sql_stmt varchar2(2048);
  v_emp_no varchar2(30);

begin
  sql_stmt := 'select emp_no from vb_emp_no partition(' || p_name || ')';
  open v_emp_cursor for sql_stmt ;

loop
  fetch v_emp_cursor into v_emp_no;
  EXIT WHEN v_emp_cursor%NOTFOUND;
  insert into employees
          (EMPLOYEE_NUMBER,
           VACATION_BALANCE,
           RATE, PERIOD, TOTAL_RATE)
  select distinct employee_number, vacation_balance, rate, period, total_rate
    from   remote_employee_view@remote_link;
  where  EMPLOYEE_NUMBER = v_emp_no;
end loop;

commit;
close v_emp_cursor;
end;
/

CREATE procedure refresh_employees
  authid definer
is
begin
  execute immediate 'truncate table vb_emp_no';
  insert into vb_emp_no
  select distinct employee_number from remote_view@db_link;
  commit;
  execute immediate 'truncate table employees';

dbms_scheduler.create_job(job_name => dbms_scheduler.generate_job_name('VB_P1_'),
   job_type => 'PLSQL_BLOCK',
   job_action => 'begin refresh_employee_data_part(''VB_EMP_NO_P1''); end;',
   comments => 'Thread 1 to refresh employees',
   enabled => true,
   auto_drop => true);

dbms_scheduler.create_job(job_name => dbms_scheduler.generate_job_name('VB_P2_'),
   job_type => 'PLSQL_BLOCK',
   job_action => 'begin refresh_employee_data_part(''VB_EMP_NO_P2''); end;',
   comments => 'Thread 2 to refresh employees',
   enabled => true,
   auto_drop => true);

dbms_scheduler.create_job(job_name => dbms_scheduler.generate_job_name('VB_P3_'),
   job_type => 'PLSQL_BLOCK',
   job_action => 'begin refresh_employee_data_part(''VB_EMP_NO_P3''); end;',
   comments => 'Thread 3 to refresh employees',
   enabled => true,
   auto_drop = true);

dbms_scheduler.create_job(job_name => dbms_scheduler.generate_job_name('VB_P4_'),
   job_type => 'PLSQL_BLOCK',
   job_action => 'begin refresh_employee_data_part(''VB_EMP_NO_P4''); end;',
   comments => 'Thread 4 to refresh employees,
   enabled => true,
   auto_drop => true);

end;
/

End Result the job finished in 50 minutes more than 4x faster.

Hazem Ameen
Senior Oracle DBA

There are 3 steps for accomplish column masking:

1. A function to be used by the policy (function policy) created in next step.
2. Use dbms_rls package to create the policy.
3. Assign “exempt access policy” to users to be excluded from the policy. These users can see all data with no masking.

Step 1: Create Function Policy

This function will be called be the policy to create the column masking. Function name can be any name you select. In my case I called vpd_function. If predicate evaluated to true, then data will to show to all users. In my case I made sure that function will always evaluate to false by looking for (rowid = 0) which will never be true. I suggest creating the function under system account.

CREATE OR REPLACE
FUNCTION vpd_function (obj_owner IN VARCHAR2, obj_name IN VARCHAR2)
RETURN VARCHAR2
AS
BEGIN
RETURN 'rowid = ''0''';
END vpd_function;
/

Step 2: Create Policy

BEGIN
DBMS_RLS.ADD_POLICY(object_schema=> 'SCOTT',
object_name=> 'EMP',
policy_name=> 'scott_emp_policy',
function_schema=> 'SYSTEM',
policy_function=> 'vpd_function',
sec_relevant_cols=> 'JOB',
policy_type => DBMS_RLS.SHARED_STATIC,
sec_relevant_cols_opt=> dbms_rls.ALL_ROWS);

END;
/

Step 3: Exclude Some Users from Policy

Users who need to see all the data without any masking need to be granted “exempt access policy”

Notes:

• Dropping a table that has a policy will drop the policy but not the function policy.
• Renaming the table will NOT drop the policy or disable it. The policy will remain active.
• Export/Import will also export/import the policy along with the table, but will not export/import the function policy.
• When exporting a table under a policy, make sure the user exporting the table has “exempt access policy” grant, otherwise, the column under the policy will always be null and column data will be lost.

Some Important Views

dba_policies
v$vpd_policy

Hazem Ameen
Senior Oracle DBA

I hope this post will answer all these questions.

SPFILE Backup

Our design goal is to have 2 backups of SPFILE: 1 on disk and the other backup on tape. This way you can first attempt to recover from disk which is faster, if you can’t then try the tape.

Here are the steps to accomplish our design (Tested in 10.2.0.4 Linux 64-bit):

1) Enable control file auto backup to disk.

Every time a change is made to control file, an auto backup will take place of both control file and SPFILE. Unfortunately SPFILE changes by itself don’t constitute an auto backup. In addition to auto backups when control file changes, every time you have a backup statement in your RMAN script (ex. backup database plus archive), an auto backup will be generated.

To enable control file auto backup:

rman> CONFIGURE CONTROLFILE AUTOBACKUP ON;

You can send your auto backups to anywhere on disk. The default is $ORACLE_HOME/dbs. I prefer sending auto backups to FRA (especially in a RAC environment). You can read about how to set up FRA and send auto backups to it here.

Here is an example of sending auto backups to a specific location on disk:

CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO ‘/u04/backups/auto/%F’;


Now you accomplished first design goal which is backup SPFILE to disk.

2) Include SPFILE in your RMAN backup sets to tape.

The default behavior when you enable auto backup is NOT to backup SPFILE in RMAN backup pieces. To overwrite this behavior, this RMAN statement will create backup piece for SPFILE and implicitly create an auto backup on disk as configured in previous step.

backup
incremental level 0
spfile format ‘spfile_%d_%s_%T.bak’  tag ‘spfile backup’
database include current controlfile format ‘data_%d_%s_%T.bak’  tag ‘data backup’
archivelog all format ‘arc_%d_%s_%T.bak’  tag ‘archive backup’ delete input;

Now we accomplished second step which is backing up SPFILE to tape with your daily rman backup.

Recovery Scenarios

Here are the most common recovery scenarios I came across, remember Oracle will not recover an SPFILE file to its original location while the database is up even if existing SPFILE is deleted. You will get this “RMAN-06564 must use the TO clause when the instance is started with SPFILE”, which is a little confusing. So simply recover to a different location and do “cp”.

A) Recover latest SPFILE while database down

set dbid <your DBID>

startup

B) Recover from auto backup in Flashback Recovery Area, database up

This is RAC/ASM with FRA placed in ASM

RMAN> restore spfile to ‘<path with filename>‘ from autobackup;
Starting restore at 11-MAR-09
allocated channel: ORA_DISK_1
channel ORA_DISK_1: sid=128 instance=raca1 devtype=DISK
recovery area destination: +ARCDG
database name (or database unique name) used for search: RACA
channel ORA_DISK_1: autobackup found in the recovery area
channel ORA_DISK_1: autobackup found: +arcdg/RACA/AUTOBACKUP/2009_03_11/s_681250817.257.681250819

restore spfile to pfile ‘<path and filename>’ from autobackup db_recovery_file_dest=’+ARCDG’ db_name=’<your db name>’;

C) Recover a specific backup of SPFILE

This is useful if you have bad parameters in your SPFILE and want to recover an older cleaner copy (SPFILE point in time recovery, now how about that )

list backup of spfile;

restore spfile to ‘<path and filename except original location when DB started> from ‘<backup piece or auto backup>’;

D) Recover an SPFILE to PFILE

Always useful if you want to read content of your parameter file.

restore spfile to pfile ‘<your path and filename>‘  from autobackup;


 

Hazem Ameen
Senior Oracle DBA

This post deals with steps you have to implement to get it working.

First, every entry in the redo log files has a tag associated with it. The default value for this tag is null. The capture process default behavior is not to check for tags at all, in other words, it will read and process every entry in the redo log regardless of it’s tag value. But what we want is for the capture process to read and process null tags only and ignore non-null tags (these are the DML or DDL that will be skipped).

So the question, how to change the capture process behavior so it will ignore non-null tags?
The answer is by adjusting the capture rules for DML & DDL (depends on whether you need to ignore both operations).

The following scripts assume you have the default Streams configuration without adding any additional custom rules.

Get Capture Rule name and rule condition.
select rule_name, rule_condition
from dba_streams_rules
where rule_set_owner = ‘STRMADMIN’
and streams_type = ‘CAPTURE’;

It should return 2 rows, 1 rule for DML and the other for DDL.

Here how the DDL capture rule looks like before changing:

((((:ddl.get_object_owner() = ‘schema name’ or :ddl.get_base_table_owner() = ‘schema name’) and :ddl.get_source_database_name() = ‘database name’ )) and (:ddl.get_compatible() <= dbms_streams.compatible_10_2))

Now add condition to capture null tags only

BEGIN
DBMS_RULE_ADM.ALTER_RULE (
rule_name => ‘CWSAPP66′,
condition => ‘((((:ddl.get_object_owner() = ”CWSAPP” or :ddl.get_base_table_owner() = ”CWSAPP”) and :ddl.get_source_database_name() = ”CWSRD” ))
and (:ddl.get_compatible() <= dbms_streams.compatible_10_2) and (:ddl.is_null_tag() = ”Y”))’,
evaluation_context => NULL);
END;
/

Now from a new session

dbms_streams.set_tag(‘01’);

DDL or DML changes will not get replicated

dbms_streams.set_tag(null); — go back to replicating everything

Remember if using bi-directional replication, you have to adjust capture rules in every site.

Hazem Ameen
Senior Oracle DBA

The person that made my wife yell was not a thief or an attacker, but your common average driver in KSA. The driving in Saudi is a combination of rude, aggressive, and in some cases flat out suicidal. It definitely gets under your skin and you respond rather erratically like.

Back in USA people are scared of drunk drivers, but in Saudi, we are fearsome of most drivers and especially of teen drivers. There are young teens that barely can see over the wheel and driving an 8-15 passenger 4×4 SUV. It came no surprise that KSA has one of the highest car accident death rates in the world.

Knowing all risks involved, would you let your wife or daughter drive (if it is ever allowed)? Even though is none of my business, but I don’t think so, not unless KSA fixes this domestic problem. Why would anybody want to take a risk on his wife’s or daughter’s life!

If you ever in KSA, here are some tips to help you cope with this issue:

Few expatriates elect not to own a car but rather relying on company’s transportation and taxis.

Some co-workers take longer and easier routes just to elude traffic and come to the job calmer.

Know the problematic routes and times of traffic congestion to avoid them.

Remind yourself to always remain calm. This is one fight you can’t really win.

Most expatriates buy 4×4 SUVs; they are safer in case car accidents (God forbid).

Drive defensively, can’t stress this point enough.

Hazem Ameen