Estimate database usage - Oracle

One question always haunts me a bit, "Could you provide the database machine requirement?".
Here's one method I often use to estimate the size for Oracle database:
1.  SELECT SUM(BYTES) FROM (
select SEGMENT_NAME, SUM(BYTES)/(1024*1024) BYTES from user_extents
where segment_type = 'TABLE'
and segment_name like 'ACT_%'
GROUP BY SEGMENT_NAME);

select SEGMENT_NAME, SUM(BYTES)/(1024*1024) BYTES from user_extents
where segment_type = 'TABLE'
and segment_name like 'ACT_%'
GROUP BY SEGMENT_NAME;

Tomcat JMX Monitoring

Monitoring Tomcat's resources could be done using the JConsole app which already included in the JDK. In order to monitor Tomcat, we also need to prepare the Tomcat first.

To configure Tomcat in *nix environment, you need to add the parameter below when starting tomcat:

-Dcom.sun.management.jmxremote
  -Dcom.sun.management.jmxremote.port=%my.jmx.port%
  -Dcom.sun.management.jmxremote.ssl=false
  -Dcom.sun.management.jmxremote.authenticate=false

To add the configuration above, one of the way is as below:
1. Open your catalina.sh
2. Add below line (example port is 12300)

3. Restart your tomcat
4. If you need to access it from other computers, make sure the firewall configuration  allows the configured port to be accessed from outside
5. In case the remote connection is not working, try to add below line in the configuration
    -Dcom.sun.management.jmxremote.host=localhost


Reference:
- https://tomcat.apache.org/tomcat-7.0-doc/monitoring.html
- http://docs.oracle.com/javase/6/docs/technotes/guides/management/jconsole.html

Resize LVM Partition in Virtual Box

Here's a note on how to resize LVM partition in VirtualBox
1. VBoxManage modifyhd vdi_path --resize new_size_in_mb
2. Resize the disk.
    a. Run the VM
    b. Open terminal/console and execute command below
        + df
           Write down the logical mapping, in my case it is /dev/mapper/vg_iplus-lv_root
     

     
        + fdisk -l
           Write down the Device name, in my case /dev/sda2

     
 + fdisk /dev/sda
     The next steps will delete the partition to resize, recreate the partition and make it primary. Then resize the partition to the new size and write the changes.
 + Still in fdisk, delete the partition
     input: d
 + Still in fdisk, select the device to partition number. In my case it is 2 (/dev/sda2)
     input: 2
   

  + Still in fdisk, create new partition and select the same partition as in previous steps (e.g. 2)
     Input: n
     Input: 2

  + Set the size for the new partition, set it to occupy all available space (or as preferred)
      Input: enter (twice to set default size)

  + Commit the changes
     Input: w

   + Reboot server
      Command: reboot

   + Resize the physical volume
      Command: pvresize /dev/sda2
     /dev/sda2 refers to the partition changed previously

   + Verify the new size, make sure the new size is applied
       Command: pvscan

   + Extend the logical volume to take all free space
      Command: lvextend -l +100%FREE /dev/mapper/fedora-root

   + Resize the file system
      Command: resize2fs /dev/mapper/vg_iplus-lv_root

   + Verify the new changes

      Command: df

And we're done! :)

Reference:
https://blog.jyore.com/2013/06/virtualbox-increase-size-of-rhelfedoracentosscientificos-guest-file-system/

Find blocking query - Oracle

If you have a query that stops and it seems that it will never end, you might have another query blocking it. Here's one help to find out which query blocking it.

SELECT s1.username || '@' || s1.machine
    || ' ( SID=' || s1.sid || ' )  is blocking '
    || s2.username || '@' || s2.machine || ' ( SID=' || s2.sid || ' ) ' AS blocking_status
    FROM v$lock l1, v$session s1, v$lock l2, v$session s2
    WHERE s1.sid=l1.sid AND s2.sid=l2.sid
    AND l1.BLOCK=1 AND l2.request > 0
    AND l1.id1 = l2.id1
    AND l1.id2 = l2.id2;

References:
http://www.oraclerecipes.com/monitoring/find-blocking-sessions/

SQLServer tips

This page would be a growing list for things related to SQLServer. Mostly kept for the rainy season :)

1. How to find most expensive queries (for cache query plans only) 

(ref: http://blog.sqlauthority.com/2010/05/14/sql-server-find-most-expensive-queries-using-dmv/)

SELECT TOP 10 SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,

((CASE qs.statement_end_offset

WHEN -1 THEN DATALENGTH(qt.TEXT)

ELSE qs.statement_end_offset

END - qs.statement_start_offset)/2)+1),

qs.execution_count,

qs.total_logical_reads, qs.last_logical_reads,

qs.total_logical_writes, qs.last_logical_writes,

qs.total_worker_time,

qs.last_worker_time,

qs.total_elapsed_time/1000000 total_elapsed_time_in_S,

qs.last_elapsed_time/1000000 last_elapsed_time_in_S,

qs.last_execution_time,

qp.query_plan

FROM sys.dm_exec_query_stats qs

CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt

CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp

ORDER BY qs.total_logical_reads DESC -- logical reads

-- ORDER BY qs.total_logical_writes DESC -- logical writes

-- ORDER BY qs.total_worker_time DESC -- CPU time

2. Queries taking longest elapsed time

SELECT TOP 10

qs.total_elapsed_time / qs.execution_count / 1000000.0 AS average_seconds,

qs.total_elapsed_time / 1000000.0 AS total_seconds,

qs.execution_count,

SUBSTRING (qt.text,qs.statement_start_offset/2,

(CASE WHEN qs.statement_end_offset = -1

THEN LEN(CONVERT(NVARCHAR(MAX), qt.text)) * 2

ELSE qs.statement_end_offset END - qs.statement_start_offset)/2) AS individual_query,

o.name AS object_name,

DB_NAME(qt.dbid) AS database_name

FROM sys.dm_exec_query_stats qs

CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) as qt

LEFT OUTER JOIN sys.objects o ON qt.objectid = o.object_id

where qt.dbid = DB_ID()

ORDER BY average_seconds DESC;

3. Queries doing most I/O

SELECT TOP 10

(total_logical_reads + total_logical_writes) / qs.execution_count AS average_IO,

(total_logical_reads + total_logical_writes) AS total_IO,

qs.execution_count AS execution_count,

SUBSTRING (qt.text,qs.statement_start_offset/2,

(CASE WHEN qs.statement_end_offset = -1

THEN LEN(CONVERT(NVARCHAR(MAX), qt.text)) * 2

ELSE qs.statement_end_offset END – qs.statement_start_offset)/2) AS individual_query,

o.name AS object_name,

DB_NAME(qt.dbid) AS database_name

FROM sys.dm_exec_query_stats qs

CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) as qt

LEFT OUTER JOIN sys.objects o ON qt.objectid = o.object_id

where qt.dbid = DB_ID()

ORDER BY average_IO DESC;

4. Find blocking and blocked queries (sessions)

Use sp_who2

Create swap file for your VPS

Just got a VPS for your own use? RAM only 512MB? or maybe you are accustomed to out of memory error and need a way to increase your memory.
Then you got to have a swap file for your VPS, the size of which will depend on your usage.

One thing to remember, swap file is created on your storage. So don't expect it to be as fast as your RAM.

Here are the steps you can follow (works on Ubuntu 12.04):
1. Check for existing swap file, if you already have one then all are good.
    swapon -s

2. Create your file for the swap file
sudo dd if=/dev/zero of=/swapfile bs=1024 count=256k

3. Set it as swap file
sudo mkswap /swapfile

4. Register the swap file
sudo swapon /swapfile

5. Make it permanent on the system
sudo nano /etc/fstab

And add the following line at the end:
/swapfile       none    swap    sw      0       0 

6. Set the swappiness
echo 10 | sudo tee /proc/sys/vm/swappiness
echo vm.swappiness = 10 | sudo tee -a /etc/sysctl.conf

7. Protect the file

sudo chown root:root /swapfile 

sudo chmod 0600 /swapfile

That's all, enjoy your VPS.

Reference:

https://www.digitalocean.com/community/tutorials/how-to-add-swap-on-ubuntu-12-04

Configure reverse ssh tunneling

When you have a need to access your local PC from the internet, you might be interested in this blog.

Requirement:
1. Root access to a public server (or any kind of access as long as you can do things below :).
2. SSH client


Now, let's begin with the server:
1. Edit sshd_config located in /etc/ssh/ and add the line below
GatewayPorts clientspecified

2. Restart sshd
sudo service ssh restart

That's all for the server.


To the local PC we go:
1. Create reverse ssh tunneling using ssh client.
Open a terminal/console to run the ssh client. In windows you might use Putty.
ssh user@server -R server:serverport:localdestination:localport

2. Test the connection by using telnet
telnet server serverport
Expect no connection refused :)


The steps above is very simple but requires some configuration changes in the public server.

There's a more secure way but requires more effort on the client side which means only geek will be able to access it :) I'll discuss it later on a separate blog.

Setting up vnc for ubuntu desktop

If your installation is ubuntu desktop and you need to remote it using vnc, the steps below might help you to do it:
1. Update your package list
sudo apt-get update

2. Install Gnome desktop environment essential component and Gnome session manager
sudo apt-get install gnome-core gnome-session-fallback

3. Install vnc server
sudo apt-get install vnc4server

4. Start the vnc server
vncserver 
Note: find out the console port being assigned to you. There is a way to start up the connection with a pre defined port number (e.g. vncserver :1)

Example: the assigned port is 1 (5901)

New 'server:1 (server)' desktop is server:1

Starting applications specified in /home/user/.vnc/xstartup

Log file is /home/user/.vnc/user:1.log

5. Edit the xstartup file located /home/user/.vnc/ as below.

At first, when you open the vnc connection there will be only a terminal. You will need to start the gnome-session for the vnc connection.

#!/bin/sh

# Uncomment the following two lines for normal desktop:

 unset SESSION_MANAGER

#exec /etc/X11/xinit/xinitrc

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup

[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources

xsetroot -solid grey

vncconfig -iconic &

#x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &

#x-window-manager &

gnome-session --session=gnome-classic &

6. Restart the vncserver by killing it first then starting it up again

vncserver -kill :1

To access vnc from the client, you need to install vnc viewer app. 

The sample url for vnc: server_url:5901

the port number is assigned when you start the vnc connection.

Up to now, for ubuntu server I use Xfce desktop environment instead of gnome. It is lightweight and somehow I still can't setup the gnome env under vnc :)

Reference link:
http://coddswallop.wordpress.com/2012/05/09/ubuntu-12-04-precise-pangolin-complete-vnc-server-setup/

Configuring php for handling large file upload

Cut it short, what I need is to configure my nginx-php to be able to handle around 200M file upload.
So here are the configuration changes I need to make:
1. nginx.conf
    usually the file is placed in /etc/nginx/nginx.conf.
    The parameters to add inside http are:
    - client_max_body_size: specifies the maximum accepted body size of a client request
    - client_body_timeout: sets the read timeout for the request body from client
    http {

        ##
        # Basic Settings
        ##

        client_max_body_size 200m;
        client_body_timeout 600s;
       
        ....
   }

2. php.ini
    Since I use php5-fpm, this file is placed in /etc/php5/fpm/php.ini
    The parameters to change is:
    - post_max_size: Sets max size of post data allowed
    - upload_max_filesize: The maximum size of an uploaded file
    - max_execution_time:This sets the maximum time in seconds a script is allowed to run before it is terminated by the parser.
    - max_input_time:  This sets the maximum time in seconds a script is allowed to parse input data, like POST and GET.


After changing those variables, restart both nginx and php5-fpm and you're done!

Reference links:
http://wiki.processmaker.com/index.php/Nginx_and_PHP-FPM_Installation
https://rtcamp.com/tutorials/php/increase-file-upload-size-limit/
http://www.radinks.com/upload/config.php
http://wiki.nginx.org/HttpCoreModule#client_body_timeout

Struts2 in Websphere 6.1

Just when someone asked you if Java really delivers what it promises, especially about compile once run anywhere. I just deployed a webapp developed using Struts 2, Spring, Hibernate in Websphere 6.1. And as expected, the deployment failed. After a few googling around and friend's advice, it turns out that I have to configure some properties for the server.

Here are the properties:

The new properties added are the last two, and here is a bit explanation on the property:
- com.ibm.ws.webcontainer.invokefilterscompatibility = true
 Quoted from ibm docs (see here)
With this custom property, the Web container calls custom servlet filters before looking for welcome files. Also, if the Web container cannot find a resource, it calls the custom servlet filters before creating a FileNotFoundException exception. This change enables the Web container to verify whether the custom servlet filters modify the path to a resource. 
In short, this property enables custom servlet filter to process http request and Struts 2 utilizes Servlet Filter.

- com.ibm.ws.webcontainer.assumefiltersuccessonsecurityerror = true
Quoted from ibm docs (see here)
When a request is received for a static file which does not exist, the web container calls defined servlet filters. If the filters do no successfully complete, a 404 error code is set. In a situation where application security is enabled, a security check is performed as part of filter invocation. Typically if the security check fails the web container considers the filters to have failed and still sets a 404 error code instead of the 401 error code that indicates the failure of a security check. The 404 error code enables the requester to access the static file without logging on.
 In short, just to be more secure this property is set to true. Anyway, this property doesn't have any thing to do with Struts 2.


To make sure everything works, don't forget to restart the server.

VirtualBox misc

A note for myself on my dealings with VirtualBox, refer to this link for complete command:

• Changing the virtual harddisk UUID (.vdi)
  VBoxManage internalcommands sethduuid /VirtualMachine/virtualmachine.vdi

• Resizing the virtual harddisk
  VBoxManage modifyhd virtualmachine.vdi --resize SIZE_MB

• Disable time synchronization
  VBoxManage setextradata LHBU "VBoxInternal/Devices/VMMDev/0/Config/GetHostTimeDisabled" "1"   

          You need to restart your vbox instance if it is running when you execute the command.
          Unrecommended workaround is:
          Guest Windows: go to Services, find VirtualBox Guest Additions service and stop it. You will find that time synch is no longer working, but I'm not sure whether other functionalities regarding virtual box - host integration are affected or no.
          Guest Linux:  /etc/init.d/vboxadd-timesync stop
          Disclaimer! Never tried the linux version, found it during googling.

Tracking Hibernate SQL query and parameters

Every developer using Hibernate might need to know the SQL it generates and the parameters being sent. Normally to have the sql printed out to console, one only have to configure hibernate property show_sql to true. However the parameters are not printed out.

Luckily for me, I found this post on configuring the necessary log level to be able to print out the sql and parameters. Here are the configuration that I used:

org.hibernate.SQL=DEBUG

org.hibernate.type=TRACE 

Expiring remote desktop certificate

Recently each time I need to connect remotely to one of the servers in my office, I always get certificate expired kind of error. As my curiosity drove me and based on the keyword "expired", I went to the server and accessed the console to check the server's date. It turned out that somehow the date and time of the server went back to 2002 (hey, I write this on 2011 :).
At first, I thought a simple update of the time on the server will fix this. Unfortunately the result was still the same. Googling around and found this link which helped me fixing the issue. Below are the steps which I executed on my computer along with some screenshots:

1. Open mmc.exe

2. On the file menu, click Add/remove snap-in
3. Click certificates, and then click add

4. Choose computer account and click Next

5. On the following dialog, as in my case, I choose another computer and put in the server name. Click Finish to continue

5. Back on the main window, expand the certificate node and choose Remote Desktop certificates.

6. On the middle section, the certificate which expired is displayed. Just delete it by right clicking and choose delete.

In short, we need to *renew* the certificate simply by deleting it and let windows recreate it later.

In case you cannot do it from your computer, you can always go to the server's console and do it there :) .... if you have the permission.....

Oracle query to find out table basic information

Query below helped me to know the current status of my tables and their places in the oracle db. So that I could further decide whether I should move it or not, notwithstanding that I must understand the undergoing code accessing those tables.

select tbl.OWNER || '.' || tbl.TABLE_NAME table_name, tbl.TABLESPACE_NAME, trunc(ds.BYTES/1024/1024) || ' Mb' table_size, tbl.NUM_ROWS, tbl.LAST_ANALYZED from SYS.ALL_TABLES tbl
join SYS.dba_segments ds on ds.SEGMENT_NAME = tbl.TABLE_NAME
where ds.SEGMENT_TYPE = 'TABLE'

The output of the query will have:
- Table name
- Tablespace where the table is stored
- The size of the table in Mb
- Number of rows
- Last analyzed

The above query could be expanded further just as needed.

How to configure the way BeanUtils converts null value

By default BeanUtils.setProperty always assign a default value for null. For an instance, if the property's type being set is Integer then you will get 0. This could be the behavior that many do not expect, for example you use Struts and you have an entry form with a number field. And you're expecting that if the user leave it empty, you would like to display it as empty later on.
BeanUtils utilized by Struts will fill the field as 0 instead of leaving it null.

Underneath BeanUtils is utilizing a ConverterUtils to convert the value before setting it to the property. Therefore to change this behavior all we need to do is to register a new IntegerConverter to ConverterUtils, thereby overriding the default converter used.

e.g.
ConvertUtils.register(new IntegerConverter(null), Integer.class);

The same thing you should do for other types as needed.

If you indeed configure the converter, you might want to read this.

Understanding seam's persistence context configuration

Actually this post is a note for myself regarding how seam could be configured in the web application. The facts are:
- The application is running on JBoss 5.1
- Seam is used to manage the lifecycle of EntityManager instances

Here's the configuration:

1. Datasource configuration file placed in {server}/deploy
   JNDI name is Datasource
2. Persistence unit configuration in persistence.xml
   <persistence-unit name="PersistenceUnit" transaction-type="JTA">
   <provider>org.hibernate.ejb.HibernatePersistence</provider>
   <jta-data-source>Datasource</jta-data-source>
   ......
   <properties>
   ........
   <property name="jboss.entity.manager.factory.jndi.name" value="java:/EntityManagerFactory"/>
   </properties>
   
   
   
3. Declaration of persistence-unit-ref element in web.xml
   <persistence-unit-ref>
   <persistence-unit-ref-name>PersistenceUnit/pu</persistence-unit-ref-name>
   <persistence-unit-name>PersistenceUnit</persistence-unit-name>
   </persistence-unit-ref>
   
4. Configuration in component.properties
   seamBootstrapsPu=false
   seamEmfRef=#{null}
   puJndiName=java:comp/env/PersistenceUnit/pu
5. Configuration of persistence managed context in components.xml
   <persistence:managed-persistence-context name="entityManager" auto-create="true"
   entity-manager-factory="@seamEmfRef@"
   persistence-unit-jndi-name="@puJndiName@"/>
   
   

Here's the short explanation:

1. A new datasource is deployed (step 1)
2. The datasource is referred to in persistence.xml (step 2)
3. Add reference to the new persistence unit name in web.xml (step 3)
4. Add new property in component.properties which refers to the persistence-unit-ref-name declared in web.xml (step 4)
5. The values configured in component.properties is used to configure the managed-persistence-context (step 5)

Note: The entity manager factory is bound to the global JNDI thru "jboss.entity.manager.factory.jndi.name" property configured during step 2, however it is not used... yet :)

Beware of customizing BeanUtils converter

If you ever have to customize BeanUtils converter just like I did, one thing that should be of your concern and that is BeanUtils is also utilized by many open source libraries. And some of the libraries might happened to have the same need as yours to customize the converters. If in any case you are using one of those libraries, it just might override your converters.

In my case, I was using jXLS . It is a great tool and it's my fault that I didn't read the documentation thoroughly. The documentation says:

"jXLS integrates with Jakarta BeanUtils conversion utilities to perform actual conversion from Excel cell values into bean properties...... BeanUtils Converters for primitive types return a default value when a conversion error occurs. jXLS overrides this behaviour in ReaderConfig class registering these classes to throw a ConversionException."

And what I configured for my application is for BeanUtils to return null for empty string instead of the default value.

Now, here's a short story on what truly happened to my application due to the conflict described above:
At first, an empty text field (for number input) in the form is converted to null and after some operations performed on the application "suddenly" it throws NumberFormatException for empty text field in the form. I'm lucky to know the steps to reproduce it and it turns out that things started to act strange after jXLS is utilized (to create/read excel file).

At this point, I'm grateful to have jXLS and BeanUtils source code. I could just debug into the source code and analyze it. And I found out that jXLS overrides BeanUtils's converter configuration as the documentation said. Now all I can do is after every invocation of jXLS I have to reconfigure BeanUtils as my application need it.

One more thing still disturb me, how to handle concurrent processes where one process is still working on jXLS and the other one need to convert the value from the form. And both are utilizing BeanUtils. I guessed at this moment I have to accept that I will get NumberFormatException and tell the user to try to submit the form again (and hoping that the jXLS process is already finished).

Setting up usb modem for ubuntu 10.10

If you want to setup the drivers for your usb modem, firstly check out this link. It shows the list of 3G hardware and the test result along with the necessary information.
In my case, I have Option Wireless iCON 225 (HSDPA) and I'm quite lucky to have mine listed as working and there's enough information for preparing the drivers.

Here's the summary of how I prepare the drivers:
1. Download and compile ozerocdoff --> udev.tar.gz (http://www.pharscape.org/ozerocdoff.html)
2. Skip installing hso driver kernel module since my kernel version is 2.6.35 (which is above or equal 2.6.31) --> http://www.pharscape.org/hso.html
3. Install HSOConnect (http://www.pharscape.org/hsoconnect.html) which requires hsolinkcontrol (http://www.peck.org.uk/hsolinkcontrol.html). And it turns out it just didn't work for me and I opt to use the default Network Manager.
4. Check if all works fine:
- Make sure hso module is already installed
find /lib/modules/`uname -r` -name 'hso.ko'
- Plug in the usb modem device and execute command below
lsmod | grep hso
If you see that hso is listed that's a good sign for you
- Check if the ports have been created
ls /dev/ttyHS*
My output is: /dev/ttyHS0 /dev/ttyHS1 /dev/ttyHS2
If you can see the output that also means ozerocdoff is already working
5. Go to my Network Manager, edit connections and add a new Mobile broadband connection.

6. Go online with my new broadband connection :)


References:

1. Network Manager - Hardware 3G (https://wiki.ubuntu.com/NetworkManager/Hardware/3G)
2. PHARscape iCon 255 (http://www.pharscape.org/icon225.html)
3. ozerocdoff (http://www.pharscape.org/forum/index.php?topic=545.0)
4. HSOConnect (http://www.pharscape.org/forum/index.php/topic,743.0.html)
5. Using Network Manager (http://www.pharscape.org/networkmanager-0.7.0-and-3g-wwan-modems.html)
   

Aspire 4741z battery is not charging

If a battery is not charging then the cause is either the battery is broken or the charging module needs replacement or could be BIOS problem. In my case the cause is neither above, apparently after taking out my battery I just realized that somehow I left the lock switch to unlock position. And indeed that's the cause of the battery is not charging.

Configuring mic in aspire 4741z - ubuntu 10.10

Luckily for me, my installed alsa is already the latest one. All that is needed is just to add 2 lines below to /etc/modprobe.d/alsa-base.conf

alias snd-card-0 snd-hda-intel
options snd-hda-intel model=acer


My mic works now but it seems a bit quiet. Hopefully I can find the way to boost the mic.

A nice tool that might help is alsamixer, run it from a terminal.