Thursday, October 22, 2015

WebLogic, OHS, and X-Forwarded- For

In our environment we run a number of WebLogic based applications behind a Load Balancer and require the Client IPs to show up in our access logs. This is accomplished in different ways depending on the applications. Our first set up is Grails running through WLS. In this case access logging is being handled through the managed server. To achieve this go to your WLS console and navigate to Environment/Servers. There pick the server that you are making the change for. From there pick the 'Logging' tab at the top and select 'HTTP' as the secondary tab. Open the Advanced options at the bottom of your page. The Format needs to be changed from Common, to Extended and at the end of the 'Extended Logging Format Fields' you will need to add the text 'cs(X-Forwarded-For)' to the end. When done it will look somewhat like this:

Hit the Save button o the bottom. Once the change is applied it will not take effect until you restart your managed server.


When we attempted to do the same in a WebLogic environment running Oracle HTTP Server this did not work. In that case it is OHS itself that is capturing the access data. To achieve the appropriate logging there we went to the EM page and clicked on our OHS instance. For the Oracle HTTP Server drop down at the top we chose the Administration>Log Configuration option. On this page press the 'Manage Log Formats' button under the Access Log section. In this page you will be presented with the current log formats, common is the default. Choose the format you want to use, or create your own, and add '%{X-Forwarded-For}i' to it where you would like to see it. For instance if you wanted it at the front of the common logging format you would end up with the following:

     %{X-Forwarded-For}i %h %l %u %t \"%r\" %>s %b 

Hit Ok on this page and if you are still using the common logging format hit apply. If you made your changes to one of the other formats then select your log, hit the edit button, and change the log format in the drop down. Hit ok to leave this box and then apply on the next page. Whichever decision you made , either the default or another logging format, you will have to restart your OHS server when done.


Wednesday, July 8, 2015

Adding outline hints to poorly performing Mat Views

I recently did an upgrade to 12.1.0.2 on one of our warehouse style databases and set the optimizer to the new level. We saw a lot of improvement for some materialized views and some degradation in full refresh time in others, some ridiculously so. We did run into an interesting situation with a couple where the refresh would take hours but the query itself only took a couple of minutes. To solve this we used the outline of the properly performing query as a hint for the materialized view.

First the query was run:

SELECT /*FINDME*/
      a.term
       a.id,
       b.subj_code
FROM term a, subj b
WHERE     a.term = b.term
       AND a.crn = b.crn;

Find the SQL_ID for the query:

select SQL_TEXT, SQL_ID from v$sql where sql_text like '%/*FINDME%';

And display the plan cursor for the SQL_ID:

select * from table(dbms_xplan.display_cursor('b0rzrxmb96ska',0,'ADVANCED ROWS ALLSTATS LAST'));

From the output copy the outline section that looks similar to the following: 

 /*+
    BEGIN_OUTLINE_DATA
    IGNORE_OPTIM_EMBEDDED_HINTS
    OPTIMIZER_FEATURES_ENABLE('12.1.0.2')
    DB_VERSION('12.1.0.2')
    ALL_ROWS
    OUTLINE_LEAF(@"SEL$2")
    MERGE(@"SEL$1")
    OUTLINE(@"SEL$1")
    FULL(@"SEL$2" "A"@"SEL$2")
    END_OUTLINE_DATA
*/

Hint the materialized view using that hint:

Create materialized view my_mv as SELECT  /*+
    BEGIN_OUTLINE_DATA
    IGNORE_OPTIM_EMBEDDED_HINTS
    OPTIMIZER_FEATURES_ENABLE('12.1.0.2')
    DB_VERSION('12.1.0.2')
    ALL_ROWS
    OUTLINE_LEAF(@"SEL$2")
    MERGE(@"SEL$1")
    OUTLINE(@"SEL$1")
    FULL(@"SEL$2" "A"@"SEL$2")
    END_OUTLINE_DATA
*/      a.term
       a.id,
       b.subj_code
FROM term a, subj b
WHERE     a.term = b.term
       AND a.crn = b.crn;

In performing the above steps I was able to bring the materialized view creation and refreshes to a time close to the query on which it was based.

*The select statement I'm using here does not necessarily match the outline as I had to change the structure of the query for publication.

  

Tuesday, May 12, 2015

Identity Columns...

In Oracle 12c the new identity column has been added. This is a column type that basically simulates a sequence and trigger to insert an auto generated number to the column and is really quite simple to use. Here is a super simple example:

 create table my_table
   (
   mt_id number generated as identity,
   name varchar2(1000),
   email varchar2(1000)
   );

This creates a 3 column table, first being our identity column and the second a varchar name, and third a varchar email address. Lets insert a few values and see what we get:

 insert into my_table(name, email) values ('Jon Doe', 'jdoe@email.zzz');
 insert into my_table(name, email) values ('Joe Blow', 'jaybee@email.zzz');

 select * from my_table;

     MT_ID NAME                 EMAIL
 ---------- -------------------- --------------------
          1 Jon Doe              jdoe@email.zzz
          2 Joe Blow             jaybee@email.zzz

Great!! Our values were inserted and the identity column was auto populated from the sequence that was made. How cool and easy is that?

We could leave it there, but it does raise a few questions. Is there uniqueness on the identity column? What happens if we insert a value for mt_id?

Looking at the constraints on our new table we see that there is  only a not null check constraint.

 select table_name, constraint_type TYPE, SEARCH_CONDITION_VC CONDITION from      user_constraints where table_name ='MY_TABLE';

 TABLE_NAME           TYPE  CONDITION
 -------------------- ----- --------------------
 MY_TABLE             C     "MT_ID" IS NOT NULL

So that's a little concerning. What happens if we insert id 3 manually:

 insert into my_table(mt_id, name, email) values ('3', 'Art Smith', 'painter@email.zzz');

 ERROR at line 1:
 ORA-32795: cannot insert into a generated always identity column

Ok... that's good, but why did it happen? Well it seems that there are a few options for defining identity columns. When specifying 'generated always' we ensure that only sequence values can be used. If we define with 'generated by default' then only when we do not provide a value will the sequence be used, even if it is a null value. If we define it with 'generated by default on null' Then only not null values will over ride the sequence, nulls will default back to the sequence number.

But wait, we didn't use any of those in our definition. The following query shows us that default seems to be 'generated always' which really forces you to define otherwise if you want to bypass the sequence values:

 SELECT table_name,  column_name,  generation_type FROM   user_tab_identity_cols WHERE    table_name = 'MY_TABLE';

TABLE_NAME    COLUMN_NAME  GENERATION
-------------------- --------------------      ----------
MY_TABLE             MT_ID                ALWAYS

Ok, on to the next questions. What if we set generation to default and insert a value that already exists? What if we insert the next value in the sequence and then call the sequence for that value? First this setup:

drop table my_table;
create table my_table
  (
  mt_id number generated by default as identity,
  name varchar2(1000),
  email varchar2(1000)
  );
  
  
 insert into my_table(name, email) values ('Jon Doe', 'jdoe@email.zzz');
 insert into my_table(name, email) values ('Joe Blow', 'jaybee@email.zzz');

Then the insert of mt_id=2 which belongs to Joe Blow:

 insert into my_table(mt_id, name, email) values ('2', 'Art Smith', 'painter@email.zzz');

 1 row created.

 select * from my_table;


     MT_ID NAME                 EMAIL

 ---------- -------------------- --------------------
         2 Art Smith            painter@email.zzz
         1 Jon Doe              jdoe@email.zzz
         2 Joe Blow             jaybee@email.zzz

Well, it looks like uniqueness is not maintained in that situation. Lets try the other way around:

   insert into my_table(mt_id, name, email) values ('3', 'Guy Homme', 'guy@email.zzz');

1 row created.

 select * from my_table;

     MT_ID NAME                 EMAIL
---------- -------------------- --------------------
         2 Art Smith            painter@email.zzz
         3 Guy Homme            guy@email.zzz
         1 Jon Doe              jdoe@email.zzz
         2 Joe Blow             jaybee@email.zzz

 insert into my_table( name, email) values ( 'Scott Tiger', 'orclguy@email.zzz');

1 row created.

 select * from my_table;

     MT_ID NAME                 EMAIL
---------- -------------------- --------------------
         2 Art Smith            painter@email.zzz
         3 Guy Homme            guy@email.zzz
         3 Scott Tiger          orclguy@email.zzz
         1 Jon Doe              jdoe@email.zzz
         2 Joe Blow             jaybee@email.zzz

No uniqueness maintained there either. Is this bad? No, not really. I would say that is working as intended, though I'm not sure I would use and Identity in such manner. I'm happy the default is to always generate and thus enforce the uniqueness. 

Now you may be asking, 'What is stopping me for setting the value of the sequence back to a value that was previously used?' 
Well, it seems that this is a system generated sequence and once created you can not alter it, even as sys. 'ORA-32793: cannot alter a system-generated sequence'  





Sunday, April 19, 2015

I had the great pleasure of attending IOUG Collaborate 15 in Las Vegas this past week and came away with so much new knowledge that I was excited to get back to work and start applying it. i also got to attend a seminar that had nothing to do with Oracle software directly. This was an panel discussion called 'Nothing to BLOG about - think again' and, from the session description, objective 1 was to 'Increase confidence required to share technical knowledge and experiences through BLOGging' . I had always felt that the things that I could Blog about were boring or had already been blogged on 1000 time. Michael Abbey, who was heading up the panel, informed me that nothing was trivial to everyone and that everyone learned in different ways,  therefore giving each blog on the same topic a chance of reaching someone in the manner they needed it. This panel gave me the confidence to try my hand at blogging, so objective 1 was accomplished. 

In the next few weeks, I'm hoping to apply the knowledge that I gained at Collaborate and document it here, if for no other reason than to have it for myself in an accessible space. If others come and read, and hopefully learn something, then all the better. Wish me luck on this new endeavor.