Showing posts with label upgrade. Show all posts
Showing posts with label upgrade. Show all posts

Tuesday, December 13, 2016

12.10.xC8 and rolling upgrades / 12.10.xC8 e rolling upgrades

Something in english about the article (original version here: http://informix-technology.blogspot.com/2016/12/1210xc8-and-rolling-upgrades-1210xc8-e.html)


English version
IBM published an alert (http://www.ibm.com/support/docview.wss?uid=swg21995897) stating that the lastest 12.10 fixpack (xC8) does not support the rolling upgrades feature. A rolling upgrade is the functionality that allows us to upgrade the secondary servers without having to restore them. This is supported only from version major.minor.xCn to major.minor.xCn+1 and when the fixpack does not change physical data structures. Unfortunately that's not the case with 12.10.xC8 as it was needed to make some internal changes to support encryption at rest (EaR).
As such it is clear that rolling upgrades can't be used to upgrade to 12.10.xC8, but that apparently was not clear on the product release notes


Versão Portuguesa
A IBM publicou um alerta (http://www.ibm.com/support/docview.wss?uid=swg21995897) informando que o último fixpack (xC8) da versão 12.10 não suporta a funcionalidade de roling upgrade. O rolling upgrade é o que nos permite fazer um upgrade dos servidores secundários sem ter de restaurar a imagem do primário e re-inicializar a replicação. Isto é suportado apenas entre fixpacks consecutivos (V.v.xCn para V.v.xCn+1 e quando o fixpack não muda estruturas físicas. Infelizmente não é o caso do 12.10.xC8 já que foi necessário mudar algumas estruturas para suportar a encriptação de dbspaces.
Como tal é claro que os rolling upgrades não podem ser usados para passar para a 12.10.xC8, mas aparentemente isso não estava claro nas notas da versão.

Friday, September 30, 2016

Sysadmin during upgrades / Sysadmin durante upgrades

Curious situation related to sysadmin during an upgrade (original version here: http://informix-technology.blogspot.com/2016/09/sysadmin-during-upgrades-sysadmin.html)


English version
Recently at a customer we were testing an upgrade from 11.50.xC7 to 12.10.xC7W1 and it apparently hanged. That's not what you like to see during an upgrade, even in a test environment.
We waited a few minutes and tried to understand what was happening. There was a thread called "convert" that apparently was idle. The thread stack trace apparently didn't change.
The functions showing up in the stack suggested it was trying to connect to a remote server following up what looks to be a call to UPDATE STATISTICS. That was a bit of a surprise.... looking at the online.log it was "hanged" during the "SCHAPI" phase. This is related to sysadmin.
Considering the upgrade didn't look good and it was a test system we decided to kill the engine and use onrestorept to revert back to version 11.50.
Then we decided to run an UPDATE STATISTICS FOR PROCEDURE on sysadmin database and voilá.... hanged again.
So, an UPDATE STATISTICS FOR PROCEDURE on sysadmin got hanged, and apparently that same thing happened during the upgrade. Further investigation led to a simple conclusion. We had created procedures in sysadmin to implement some monitoring tasks. And some of them tried to do remote operations on an engine where we centralize the monitoring data. Just before the upgrade a change was made to the instance $INFORMIXSQLHOSTS so that we "isolated" it from the outside world. This was done by adding some value to the service ports. The consequence of this was that during the upgrade, when it tried to run the UPDATE STATISTICS FOR PROCEDURE it tried to access a non-reachable remote engine. As it was not possible, it hanged some time for each remote instruction. Later tests on 12.10 showed the time it waits for each connection attempt is by default 60s and can be defined by changing the relatively new ONCONFIG parameter INFORMIXCONTIME

This can be easily reproduced. On a version 11.50 engine we create a simple (dumb) procedure:


bica@primary:informix-> onstat -

IBM Informix Dynamic Server Version 11.50.FC7 -- On-Line -- Up 00:30:33 -- 269068 Kbytes


bica@primary:informix-> cat test_remote_proc_stats.sql


CREATE PROCEDURE test_updstats_remote()

DEFINE i INTEGER;

SELECT 1 INTO i FROM stores@test:systables WHERE tabid = 1;

SELECT 2 INTO i FROM somedb@test:systables WHERE tabid = 1;

END PROCEDURE;
bica@primary:informix-> 



bica@primary:informix-> dbaccess sysadmin test_remote_proc_stats.sql

Database selected.

Routine created.

Database closed.

bica@primary:informix->

Database server "test" exists in the $INFORMIXSQLHOSTS, but points to a wrong port. Let's try an upgrade to 12.10.FC7 and see what happens.

From the online.log we get:


01:30:52 IBM Informix Dynamic Server Started.
01:30:52 Requested shared memory segment size rounded from 4308KB to 4796KB

Wed Aug 31 01:30:54 2016

01:30:54 Requested shared memory segment size rounded from 221001KB to 221004KB
01:30:54 Successfully added a bufferpool of page size 2K.

01:30:54 Event alarms enabled. ALARMPROG = '/usr/informix/srvr1210fc7w1/etc/alarmprogram.sh'
01:30:54 Booting Language from module <>
01:30:54 Loading Module
01:30:54 Booting Language from module <>
01:30:54 Loading Module
01:30:59 DR: DRAUTO is 0 (Off)
01:30:59 DR: ENCRYPT_HDR is 0 (HDR encryption Disabled)
01:30:59 Event notification facility epoll enabled.
01:31:00 Trusted host cache successfully built:/etc/hosts.equiv.
01:31:00 CCFLAGS2 value set to 0x200
01:31:00 SQL_FEAT_CTRL value set to 0x8008
01:31:00 SQL_DEF_CTRL value set to 0x4b0
01:31:00 IBM Informix Dynamic Server Version 12.10.FC7W1 Software Serial Number AAA#B000000
01:31:00 Conversion from version 11.50.xC6 Started

[...]

01:31:05 processing sysmaster
01:31:05 *** entity_ver=31 targetVersion=36 indexVersion= 0 ***
01:31:05 *** FromVersion=10 ToVersion=26 entity_ver=31 targetVersion=36 indexVersion=0 ***
01:31:05 *** FromVersion=26 ToVersion=28 entity_ver=31 targetVersion=36 indexVersion=1 ***
01:31:05 *** FromVersion=28 ToVersion=29 entity_ver=31 targetVersion=36 indexVersion=2 ***
01:31:05 *** FromVersion=29 ToVersion=30 entity_ver=31 targetVersion=36 indexVersion=3 ***
01:31:05 *** FromVersion=30 ToVersion=31 entity_ver=31 targetVersion=36 indexVersion=4 ***
01:31:05 *** FromVersion=31 ToVersion=32 entity_ver=31 targetVersion=36 indexVersion=5 ***
01:31:05 *** SQL 1 11.70.xC1 ***
01:31:05 Converting database sysmaster ...
01:31:06 The database sysmaster has been converted successfully.
01:31:06 *** FromVersion=32 ToVersion=33 entity_ver=32 targetVersion=36 indexVersion=6 ***
01:31:06 *** SQL 1 11.70.xC3 ***
01:31:06 Converting database sysmaster ...
01:31:06 The database sysmaster has been converted successfully.
01:31:06 *** FromVersion=33 ToVersion=34 entity_ver=33 targetVersion=36 indexVersion=7 ***
01:31:06 *** SQL 1 12.10.xC1 ***
01:31:06 Converting database sysmaster ...
01:31:06 The database sysmaster has been converted successfully.
01:31:06 *** FromVersion=34 ToVersion=35 entity_ver=34 targetVersion=36 indexVersion=8 ***
01:31:06 *** SQL 1 12.10.xC4 ***
01:31:06 Converting database sysmaster ...
01:31:06 The database sysmaster has been converted successfully.
01:31:06 *** FromVersion=35 ToVersion=36 entity_ver=35 targetVersion=36 indexVersion=9 ***
01:31:06 *** SQL 1 12.10.xC4 ***
01:31:06 processing sysutils

[...]

01:31:17 Conv/rev: calling conversion to 12.10.xC1 ...
01:31:19 Conv/rev: blade conversion to 12.10.xC1 completed
01:31:19 *** FromVersion=6 ToVersion=7 entity_ver=6 targetVersion=7 indexVersion=6 ***
01:31:19 *** BLADE 1 12.10.xC4 ***
01:31:19 Conv/rev: calling conversion to 12.10.xC4 ...
01:31:21 Conv/rev: blade conversion to 12.10.xC4 completed
01:31:21 Conv/rev: Finished conversion for component BLADE
01:31:21 Conv/rev: Started conversion for component SCHAPI
01:31:21 *** numVersions=7 targetVersion=7 ***
01:31:21 *** entity_ver=3 targetVersion=7 indexVersion= 0 ***
01:31:21 *** FromVersion=0 ToVersion=1 entity_ver=3 targetVersion=7 indexVersion=0 ***
01:31:21 *** FromVersion=1 ToVersion=2 entity_ver=3 targetVersion=7 indexVersion=1 ***
01:31:21 *** FromVersion=2 ToVersion=3 entity_ver=3 targetVersion=7 indexVersion=2 ***
01:31:21 *** FromVersion=3 ToVersion=4 entity_ver=3 targetVersion=7 indexVersion=3 ***
01:31:21 *** SCHAPI 1 11.70.xC1 ***
01:31:21 calling schapi_cvr1170_convert
01:31:21 *** FromVersion=4 ToVersion=5 entity_ver=4 targetVersion=7 indexVersion=4 ***
01:31:21 *** SCHAPI 1 11.70.xC3 ***



and it hangs...
Looking into the threads and thread stack:


IBM Informix Dynamic Server Version 12.10.FC7W1 -- Single-User -- Up 00:00:54 -- 266648 Kbytes

Threads:
tid tcb rstcb prty status vp-class name
2 4579d9e8 0 1 IO Idle 3lio* lio vp 0
3 458fd368 0 1 IO Idle 4pio* pio vp 0
4 4591e368 0 1 IO Idle 5aio* aio vp 0
5 4593f368 1d43600 1 IO Idle 6msc* msc vp 0
6 45970368 0 1 IO Idle 7fifo* fifo vp 0
7 45991608 44c1e028 3 sleeping secs: 1 1cpu main_loop()
8 459ed4c0 0 1 running 8soc* soctcppoll
9 459edc98 0 2 sleeping forever 1cpu* soctcplst
10 45a3cc88 44c1e8f0 1 sleeping secs: 1 1cpu flush_sub(0)
11 45a59028 44c1f1b8 1 sleeping secs: 1 1cpu flush_sub(1)
12 45a59368 44c1fa80 1 sleeping secs: 1 1cpu flush_sub(2)
13 45a596a8 44c20348 1 sleeping secs: 1 1cpu flush_sub(3)
14 45a599e8 44c20c10 1 sleeping secs: 1 1cpu flush_sub(4)
15 45a59d28 44c214d8 1 sleeping secs: 1 1cpu flush_sub(5)
16 45ab70e0 44c21da0 1 sleeping secs: 1 1cpu flush_sub(6)
17 45ab7420 44c22668 1 sleeping secs: 1 1cpu flush_sub(7)
18 45ab7760 0 3 IO Idle 1cpu* kaio
19 45c0c2c8 44c22f30 3 sleeping secs: 1 1cpu convert
20 45c0c9f8 44c237f8 2 sleeping secs: 1 1cpu aslogflush
21 45ccc958 44c240c0 1 sleeping secs: 8 1cpu btscanner_0
22 45ceba00 44c24988 3 cond wait ReadAhead 1cpu readahead_0
23 45d0eb50 44c25250 3 sleeping secs: 1 1cpu auto_tune
27 45d8aa80 44c25b18 1 sleeping secs: 0 1cpu sbspclean
28 45d30aa8 44c26ca8 1 sleeping secs: 1 1cpu defragclean
30 45d738f0 44c27570 3 sleeping secs: 1 1cpu* onmode_mon
31 45d89028 44c27e38 3 sleeping secs: 1 1cpu periodic
32 45d89608 44c28700 3 sleeping forever 1cpu memory
33 45d89be8 44c28fc8 3 sleeping secs: 15 1cpu session_mgr
35 45e08768 44c263e0 1 cond wait bp_cond 1cpu bf_priosweep()

IBM Informix Dynamic Server Version 12.10.FC7W1 -- Single-User -- Up 00:01:03 -- 266648 Kbytes

Stack for thread: 19 convert
base: 0x0000000045c19000
len: 69632
pc: 0x00000000013eaa6f
tos: 0x0000000045c24d50
state: sleeping
vp: 1

0x00000000013eaa6f (oninit) yield_processor_svp
0x00000000013ed5d3 (oninit) mt_yield
0x0000000001501340 (oninit) net_wait_for_connect
0x0000000001506ade (oninit) connsocket
0x000000000155f3c0 (oninit) tlConnect
0x000000000155c56a (oninit) slSQIreq
0x0000000001555343 (oninit) pfConReq
0x000000000154c628 (oninit) cmReqSync
0x000000000154d7ef (oninit) cmConReq
0x00000000014ee62e (oninit) ASF_Call
0x000000000076fd94 (oninit) dd_newconnect
0x00000000007712d0 (oninit) dd_remconnect
0x0000000000765404 (oninit) remcatalog
0x0000000000734831 (oninit) readtab
0x000000000073a372 (oninit) ddgettable_hierarchy_trace
0x00000000006bd1d5 (oninit) aud_ddgettable_trace
0x0000000000601206 (oninit) gettab_trace
0x0000000000655f15 (oninit) tlgetddtab_trace
0x0000000000656530 (oninit) tlp2tnode
0x000000000065af3b (oninit) tbl2tab
0x000000000063785d (oninit) sel2cb
0x0000000000639498 (oninit) subq2cb
0x00000000006632c1 (oninit) sq2expr
0x0000000000663373 (oninit) subq2expr
0x000000000066cca4 (oninit) expr2expr
0x0000000000682764 (oninit) ip_exprparse
0x00000000006a28fe (oninit) cmp_stmt
0x00000000006a2c1d (oninit) pr_compile
0x00000000006bf021 (oninit) aud_pr_compile
0x00000000006a3987 (oninit) updprocstat
0x00000000005cfba6 (oninit) excommand
0x0000000000f7eed7 (oninit) admin_exec_cmd
0x0000000000f7f2d5 (oninit) run_sql_script
0x0000000000f7f418 (oninit) run_script_etc
0x0000000000f9b826 (oninit) run_script_conv
0x0000000000f9b936 (oninit) ph_convrev
0x0000000000f42233 (oninit) schapi_cvr1170XC3_convert
0x0000000000f3f906 (oninit) do_each_version_work
0x0000000000f402d3 (oninit) generic_fw_main
0x0000000000f40cf6 (oninit) convertSM
0x00000000013c3720 (oninit) th_init_initgls
0x000000000142a518 (oninit) startup




The stack trace is clear. And precisely 2 minutes later it continues:


01:31:21 *** SCHAPI 1 11.70.xC3 ***
01:33:22 *** FromVersion=5 ToVersion=6 entity_ver=5 targetVersion=7 indexVersion=5 ***
01:33:22 *** SCHAPI 1 12.10.xC1 ***
01:33:22 calling schapi_cvr1210_convert
01:33:22 *** FromVersion=6 ToVersion=7 entity_ver=6 targetVersion=7 indexVersion=6 ***
01:33:22 *** SCHAPI 1 12.10.xC4 ***
01:33:22 calling schapi_cvr1210XC4_convert
01:33:23 Dynamically allocated new virtual shared memory segment (size 8192KB)
01:33:23 Memory sizes:resident:4796 KB, virtual:270044 KB, no SHMTOTAL limit
01:33:23 Conv/rev: Finished conversion for component SCHAPI




And it completed the conversion. In the customer situation, a later test showed a "hang" period of around 45m. So, a big warning.... The conversion process will run an UPDATE STATISTICS FOR PROCEDURE on sysadmin database. If you have created some procedures that do remote/distributed operations than you must consider this "problem" during the upgrade. Here are your options:

  1. Allow connections to that remote server(s)
  2. Remove the procedures and recreate them
  3. Use INFORMIXCONTIME parameter (12.10 only) to reduce the timeouts
  4. Remove the remote INFORMIXSERVER from the INFORMIXSQLHOSTS during migration. This also makes the connections attempts very quickly (as it simply can't find where to connect)


Versão Portuguesa
Há algum tempo num cliente, estando a testar uma migração de uma versão 11.50.FC7 para 12.10.FC7W1 deparámo-nos com um aparente bloqueio. Não é o que gostamos de ver, ainda que seja num ambiente de testes.
Esperámos uns minutos e tentámos perceber o que se estava a passar. Havia uma thread chamada "convert" que aparentemente estava inativa. O seu stack trace não parecia mudar
As funções que apareciam no stack sugeriam que estava a tentar contactar um servidor remoto, seguindo o que parecia ser uma chamada a um UPDATE STATISTICS. Isto foi uma grande surpresa.... Olhando para o online.log estava parado na fase "SCHAPI". Isto está relacionado com a sysadmin.
Considerando que o upgrade não estava com bom aspeto e que se tratava de um ambiente de testes decidimos abortar e usámos o onrestorept para reverter para a versão 11.50 (umas das minhas funcionalidades favoritas).
Para efeitos de teste e investigação decidimos executar um UPDATE STATISTICS FOR PROCEDURE na base de dados sysadmin. E voilá, preso de novo.
Portanto um UPDATE STATISTICS FOR PROCEDURE na sysadmin bloqueava o sistema e isso teria acontecido durante o upgrade. Após mais investigação concluímos que tínhamos criado procedimentos nesta base de dados para implementar algumas tarefas de monitorização e que esses procedimentos executavam algumas operações remotas para enviar a informação de monitorização para um sistema central. E antes do upgrade tinha sido feito uma alteração "de rotina" que consistiu na alteração do $INFORMIXSQLHOSTS de forma a "isolar" este motor. Basicamente foi adicionado um valor a todas as portas dos listeners, o que faz com que os clientes (não alterados) não consigam chegar ao ambiente. A consequência disto é que quando efetuamos o UPDATE STATISTICS este motor tenta aceder ao(s) motores referenciados nas operações remotas o que também não será possível. E como tal temos um timeout por cada operação remota encontrada nos procedimentos. Testes posteriores na 12.10 mostraram que a espera por cada tentativa de conexão é o valor por omissão (60s) ou o que estiver definido no parâmetro de $ONCONFIG INFORMIXCONTIME introduzido recentemente.

Tudo isto pode ser reproduzido facilmente. Num motor com a versão 11.50 criamos um procedimento simples (inútil):


bica@primary:informix-> onstat -

IBM Informix Dynamic Server Version 11.50.FC7 -- On-Line -- Up 00:30:33 -- 269068 Kbytes


bica@primary:informix-> cat test_remote_proc_stats.sql


CREATE PROCEDURE test_updstats_remote()

DEFINE i INTEGER;

SELECT 1 INTO i FROM stores@test:systables WHERE tabid = 1;

SELECT 2 INTO i FROM somedb@test:systables WHERE tabid = 1;

END PROCEDURE;
bica@primary:informix-> 



bica@primary:informix-> dbaccess sysadmin test_remote_proc_stats.sql

Database selected.

Routine created.

Database closed.

bica@primary:informix->

O servidor de base de dados "test" existe no $INFORMIXSQLHOSTS, mas aponta para um serviço não existente. Tentemos um upgrade para a 12.10.FC7 e vejamos o que acontece:

No online.log temos:


01:30:52 IBM Informix Dynamic Server Started.
01:30:52 Requested shared memory segment size rounded from 4308KB to 4796KB

Wed Aug 31 01:30:54 2016

01:30:54 Requested shared memory segment size rounded from 221001KB to 221004KB
01:30:54 Successfully added a bufferpool of page size 2K.

01:30:54 Event alarms enabled. ALARMPROG = '/usr/informix/srvr1210fc7w1/etc/alarmprogram.sh'
01:30:54 Booting Language from module <>
01:30:54 Loading Module
01:30:54 Booting Language from module <>
01:30:54 Loading Module
01:30:59 DR: DRAUTO is 0 (Off)
01:30:59 DR: ENCRYPT_HDR is 0 (HDR encryption Disabled)
01:30:59 Event notification facility epoll enabled.
01:31:00 Trusted host cache successfully built:/etc/hosts.equiv.
01:31:00 CCFLAGS2 value set to 0x200
01:31:00 SQL_FEAT_CTRL value set to 0x8008
01:31:00 SQL_DEF_CTRL value set to 0x4b0
01:31:00 IBM Informix Dynamic Server Version 12.10.FC7W1 Software Serial Number AAA#B000000
01:31:00 Conversion from version 11.50.xC6 Started

[...]

01:31:05 processing sysmaster
01:31:05 *** entity_ver=31 targetVersion=36 indexVersion= 0 ***
01:31:05 *** FromVersion=10 ToVersion=26 entity_ver=31 targetVersion=36 indexVersion=0 ***
01:31:05 *** FromVersion=26 ToVersion=28 entity_ver=31 targetVersion=36 indexVersion=1 ***
01:31:05 *** FromVersion=28 ToVersion=29 entity_ver=31 targetVersion=36 indexVersion=2 ***
01:31:05 *** FromVersion=29 ToVersion=30 entity_ver=31 targetVersion=36 indexVersion=3 ***
01:31:05 *** FromVersion=30 ToVersion=31 entity_ver=31 targetVersion=36 indexVersion=4 ***
01:31:05 *** FromVersion=31 ToVersion=32 entity_ver=31 targetVersion=36 indexVersion=5 ***
01:31:05 *** SQL 1 11.70.xC1 ***
01:31:05 Converting database sysmaster ...
01:31:06 The database sysmaster has been converted successfully.
01:31:06 *** FromVersion=32 ToVersion=33 entity_ver=32 targetVersion=36 indexVersion=6 ***
01:31:06 *** SQL 1 11.70.xC3 ***
01:31:06 Converting database sysmaster ...
01:31:06 The database sysmaster has been converted successfully.
01:31:06 *** FromVersion=33 ToVersion=34 entity_ver=33 targetVersion=36 indexVersion=7 ***
01:31:06 *** SQL 1 12.10.xC1 ***
01:31:06 Converting database sysmaster ...
01:31:06 The database sysmaster has been converted successfully.
01:31:06 *** FromVersion=34 ToVersion=35 entity_ver=34 targetVersion=36 indexVersion=8 ***
01:31:06 *** SQL 1 12.10.xC4 ***
01:31:06 Converting database sysmaster ...
01:31:06 The database sysmaster has been converted successfully.
01:31:06 *** FromVersion=35 ToVersion=36 entity_ver=35 targetVersion=36 indexVersion=9 ***
01:31:06 *** SQL 1 12.10.xC4 ***
01:31:06 processing sysutils

[...]

01:31:17 Conv/rev: calling conversion to 12.10.xC1 ...
01:31:19 Conv/rev: blade conversion to 12.10.xC1 completed
01:31:19 *** FromVersion=6 ToVersion=7 entity_ver=6 targetVersion=7 indexVersion=6 ***
01:31:19 *** BLADE 1 12.10.xC4 ***
01:31:19 Conv/rev: calling conversion to 12.10.xC4 ...
01:31:21 Conv/rev: blade conversion to 12.10.xC4 completed
01:31:21 Conv/rev: Finished conversion for component BLADE
01:31:21 Conv/rev: Started conversion for component SCHAPI
01:31:21 *** numVersions=7 targetVersion=7 ***
01:31:21 *** entity_ver=3 targetVersion=7 indexVersion= 0 ***
01:31:21 *** FromVersion=0 ToVersion=1 entity_ver=3 targetVersion=7 indexVersion=0 ***
01:31:21 *** FromVersion=1 ToVersion=2 entity_ver=3 targetVersion=7 indexVersion=1 ***
01:31:21 *** FromVersion=2 ToVersion=3 entity_ver=3 targetVersion=7 indexVersion=2 ***
01:31:21 *** FromVersion=3 ToVersion=4 entity_ver=3 targetVersion=7 indexVersion=3 ***
01:31:21 *** SCHAPI 1 11.70.xC1 ***
01:31:21 calling schapi_cvr1170_convert
01:31:21 *** FromVersion=4 ToVersion=5 entity_ver=4 targetVersion=7 indexVersion=4 ***
01:31:21 *** SCHAPI 1 11.70.xC3 ***



e bloqueia...
Vejamos as threads e o respetivo stack:


IBM Informix Dynamic Server Version 12.10.FC7W1 -- Single-User -- Up 00:00:54 -- 266648 Kbytes

Threads:
tid tcb rstcb prty status vp-class name
2 4579d9e8 0 1 IO Idle 3lio* lio vp 0
3 458fd368 0 1 IO Idle 4pio* pio vp 0
4 4591e368 0 1 IO Idle 5aio* aio vp 0
5 4593f368 1d43600 1 IO Idle 6msc* msc vp 0
6 45970368 0 1 IO Idle 7fifo* fifo vp 0
7 45991608 44c1e028 3 sleeping secs: 1 1cpu main_loop()
8 459ed4c0 0 1 running 8soc* soctcppoll
9 459edc98 0 2 sleeping forever 1cpu* soctcplst
10 45a3cc88 44c1e8f0 1 sleeping secs: 1 1cpu flush_sub(0)
11 45a59028 44c1f1b8 1 sleeping secs: 1 1cpu flush_sub(1)
12 45a59368 44c1fa80 1 sleeping secs: 1 1cpu flush_sub(2)
13 45a596a8 44c20348 1 sleeping secs: 1 1cpu flush_sub(3)
14 45a599e8 44c20c10 1 sleeping secs: 1 1cpu flush_sub(4)
15 45a59d28 44c214d8 1 sleeping secs: 1 1cpu flush_sub(5)
16 45ab70e0 44c21da0 1 sleeping secs: 1 1cpu flush_sub(6)
17 45ab7420 44c22668 1 sleeping secs: 1 1cpu flush_sub(7)
18 45ab7760 0 3 IO Idle 1cpu* kaio
19 45c0c2c8 44c22f30 3 sleeping secs: 1 1cpu convert
20 45c0c9f8 44c237f8 2 sleeping secs: 1 1cpu aslogflush
21 45ccc958 44c240c0 1 sleeping secs: 8 1cpu btscanner_0
22 45ceba00 44c24988 3 cond wait ReadAhead 1cpu readahead_0
23 45d0eb50 44c25250 3 sleeping secs: 1 1cpu auto_tune
27 45d8aa80 44c25b18 1 sleeping secs: 0 1cpu sbspclean
28 45d30aa8 44c26ca8 1 sleeping secs: 1 1cpu defragclean
30 45d738f0 44c27570 3 sleeping secs: 1 1cpu* onmode_mon
31 45d89028 44c27e38 3 sleeping secs: 1 1cpu periodic
32 45d89608 44c28700 3 sleeping forever 1cpu memory
33 45d89be8 44c28fc8 3 sleeping secs: 15 1cpu session_mgr
35 45e08768 44c263e0 1 cond wait bp_cond 1cpu bf_priosweep()

IBM Informix Dynamic Server Version 12.10.FC7W1 -- Single-User -- Up 00:01:03 -- 266648 Kbytes

Stack for thread: 19 convert
base: 0x0000000045c19000
len: 69632
pc: 0x00000000013eaa6f
tos: 0x0000000045c24d50
state: sleeping
vp: 1

0x00000000013eaa6f (oninit) yield_processor_svp
0x00000000013ed5d3 (oninit) mt_yield
0x0000000001501340 (oninit) net_wait_for_connect
0x0000000001506ade (oninit) connsocket
0x000000000155f3c0 (oninit) tlConnect
0x000000000155c56a (oninit) slSQIreq
0x0000000001555343 (oninit) pfConReq
0x000000000154c628 (oninit) cmReqSync
0x000000000154d7ef (oninit) cmConReq
0x00000000014ee62e (oninit) ASF_Call
0x000000000076fd94 (oninit) dd_newconnect
0x00000000007712d0 (oninit) dd_remconnect
0x0000000000765404 (oninit) remcatalog
0x0000000000734831 (oninit) readtab
0x000000000073a372 (oninit) ddgettable_hierarchy_trace
0x00000000006bd1d5 (oninit) aud_ddgettable_trace
0x0000000000601206 (oninit) gettab_trace
0x0000000000655f15 (oninit) tlgetddtab_trace
0x0000000000656530 (oninit) tlp2tnode
0x000000000065af3b (oninit) tbl2tab
0x000000000063785d (oninit) sel2cb
0x0000000000639498 (oninit) subq2cb
0x00000000006632c1 (oninit) sq2expr
0x0000000000663373 (oninit) subq2expr
0x000000000066cca4 (oninit) expr2expr
0x0000000000682764 (oninit) ip_exprparse
0x00000000006a28fe (oninit) cmp_stmt
0x00000000006a2c1d (oninit) pr_compile
0x00000000006bf021 (oninit) aud_pr_compile
0x00000000006a3987 (oninit) updprocstat
0x00000000005cfba6 (oninit) excommand
0x0000000000f7eed7 (oninit) admin_exec_cmd
0x0000000000f7f2d5 (oninit) run_sql_script
0x0000000000f7f418 (oninit) run_script_etc
0x0000000000f9b826 (oninit) run_script_conv
0x0000000000f9b936 (oninit) ph_convrev
0x0000000000f42233 (oninit) schapi_cvr1170XC3_convert
0x0000000000f3f906 (oninit) do_each_version_work
0x0000000000f402d3 (oninit) generic_fw_main
0x0000000000f40cf6 (oninit) convertSM
0x00000000013c3720 (oninit) th_init_initgls
0x000000000142a518 (oninit) startup




O stack é claro. E precisamente 2 minutos depois a conversão continua (o procedimento tinha duas instruções remotas):


01:31:21 *** SCHAPI 1 11.70.xC3 ***
01:33:22 *** FromVersion=5 ToVersion=6 entity_ver=5 targetVersion=7 indexVersion=5 ***
01:33:22 *** SCHAPI 1 12.10.xC1 ***
01:33:22 calling schapi_cvr1210_convert
01:33:22 *** FromVersion=6 ToVersion=7 entity_ver=6 targetVersion=7 indexVersion=6 ***
01:33:22 *** SCHAPI 1 12.10.xC4 ***
01:33:22 calling schapi_cvr1210XC4_convert
01:33:23 Dynamically allocated new virtual shared memory segment (size 8192KB)
01:33:23 Memory sizes:resident:4796 KB, virtual:270044 KB, no SHMTOTAL limit
01:33:23 Conv/rev: Finished conversion for component SCHAPI




E termina a conversão. Na situação concreta do cliente, um novo teste com a mesma instância apresentou um "bloqueio" durante cerca de 45m (correspondente a sensivelmente o mesmo número de instruções remotas nos procedimentos da sysadmin). Portanto um grande alerta...: O processo de conversão irá executar um UPDATE STATISTICS FOR PROCEDURE na base de dados sysadmin (isto não está documentado, mas é natural tendo em conta o processo de conversão da mesma). Se porventura criou procedimentos que acedem a instâncias remotas, então deverá ter isto em conta As opções para lidar com isto serão::
  1. Permitir as ligações ao(s) servidor remoto
  2. Remover os procedimentos e recriá-los depois da migração
  3. Usar o parâmetro INFORMIXCONTIME (a partir da 12.10) para reduzir os tempos de espera
  4. Remover o INFORMIXSERVER remoto do $INFORMIXSQLHOSTS durante a conversão. Isto também permite acelerar as tentativas de conexão pois ao não encontrar a referência no ficheiro o motor não sabe onde se ligar, e portanto não se aplicam os timeouts

Friday, September 09, 2011

Get pending In Place ALTERs / Obter as tabelas com InPlace ALTERs pendentes

This article is written in English and Portuguese
Este artigo está escrito em Inglês e Português


English version:


Introduction

The topic of in place ALTERed tables has always been present in Informix. We say a table was ALTERed in place when an ALTER TABLE instruction resulted in a new table schema, but the table was not physically changed. This is very helpful on large and busy tables. It means you can do the ALTER TABLE quickly (instantaneously) and with minimum immediate impact on system resources. Internally Informix does a very simple thing from the user point of view, but that can be complex from the engine point of view: A new version of the table definition is created, and from there on, any instruction (DML) affecting the data will use the new version. The table's existing pages are left on the older version but each time we SELECT them, the engine converts the row(s) to the new format. Additionally if we update a row, the page where it is stored will be written in the new format.

For example, if we have a table with lots of data pages, and we add a column, this will do an in place ALTER. Certain types of ALTERs don't allow in place ALTERs. For example, if we change a CHAR(5) column to a SMALLINT, this will be a slow ALTER. This usually happens when the system cannot immediately guarantee that the existing data can be stored in the new representation.

When the engine decides that it can do an in place ALTER, we don't have the option to inhibit it. Meaning the ALTER table will be done as an in place ALTER, and if we want to force the physical changes, we must do what we usually call a dummy UPDATE: UPDATE tabname SET column = column;

Impacts of in place ALTERs

There are several impacts on having tables with in place ALTERs. The immediate one is that your updates will be slightly slower, since the whole page will be converted and written. Note that this does not necessarily means higher I/O load, since the page is the smallest unit written by the engine. In other words, even if your table does not have more than one version of it's definition, when we change a single row of data, a whole page (at least) will be written. But the other rows of the page don't need to be changed. And changing an older version can mean your data will not fit on one page after conversion. So there can be more I/O.
Another potential issue is that like with any other part of the engine, the mechanism of in place ALTER can have issues. This is an overrated aspect, but it's true we've seen problems that only affect tables with in place ALTERs.

But the real issue with in place ALTERs refers to upgrades and downgrades. Through versions there have been great controversy regarding the real impact of doing in-place upgrades (conversions) with tables with pending in-place ALTERs. I've searched through the migration guides for several versions (7.3, 9.2, 9.4, 10, 11.1, 11.5 and 11.7) and they're almost completely consistent: You can upgrade with pending in-place ALTERed tables, but you cannot revert. The only exception I've found was the manual for 9.4 which states that you must remove the pending in-place ALTERs to run the conversion (upgrade) successfully.

The reason for allowing upgrades but not downgrades is pretty simple and acceptable: Informix guarantees that version N+1 can handle all previous situations of in-place ALTERs done in version N or older. But, since each version may have added new situations where in-place ALTERs can be done, we can't risk porting a pending ALTER to an older version that may not be able to handle it. Let me remind you that an in-place ALTER requires the ability to convert from one row version to a newer one (which can have more columns, or different data types etc.)

At the time of this writing I could not verify if the exception in the migration guide of version 9.4 was justified or simply a documentation problem. But the fact is that most people assume they must complete the ALTER tables that were done in-place before converting. By this I mean that they must eliminate all the pages in older versions. A valid reason to do that is that in case you need to revert you don't have to waste time running the dummy updates. Typically, if you have any issue in the new version and need to revert, you'll want to do it as soon as possible. As such, eliminating the pending inplace ALTERs before you upgrade can save you precious time if you really need to revert. In any case, the migration guide for version 11.7 clearly states that you only need to remove the pending inplace ALTERs, if they were caused after the conversion to 11.7. Any previous one (which already exists in the old version) would not need to be eliminated.
As a side note, let me state that the only time I've done reversions was during a customer and partner workshop where we were demoing this functionality... I never had to do it on a real customer situation. In any case the other limitations for reversion can be real challenges, so the pending in-place ALTERs wouldn't be your biggest concern.

Finding in-place alters

Now that we've seen in which situations we should remove pending inplace ALTERs (tables with pages in older versions), we come to another issue that is frequently asked, and to which there are several answers. Most of them are controversial or badly explained which again raises a lot of confusion. The issue is: How do we find tables with pending inplace ALTERs?
There was a discussion in the IIUG forum near the end of 2010 that focused on this issue. As usual, there were three answers to this:

  1. A quick way (based on SQL and SMI that tells us which tables suffered an inplace ALTER, but doesn't show which tables have pending inplace ALTERs. This means that unless you completely rebuild the table, just running dummy UPDATEs will not prevent the table from always appearing in the list generated by this method
  2. A slow way that's based in the oncheck -pT output (this effectively tells you the number of pages in each existing version This method will give you just the tables with pending inplace ALTERs
  3. A technical support tool that searches the tables metadata and can provide the answer pretty quickly. Only problem is that it's not generally available

And this was my motivation to do some research on this issue. After some information exchange with Andreas Legner from IBM technical support in Germany, I was able to create an SQL script that can find the tables with pending inplace alters. The script can give us the details about the number of pages in each version, it returns the database and table name, the version and number of pages it contains.

The great thing about this script is that it's fast (from a few seconds to a few minutes for very large databases) and it really shows you the current status. Contrary to what the option 1) above does, if you do the dummy updates on one table, that table will not show up if you run it again. One warning: If you test this, after the dummy update you need to force a checkpoint. The script goes through the partition headers (because the required info is stored there), and after the dummy updates the partition headers are written to disk only at checkpoint time.

The script comes in the form of a stored procedure and is based only on the sysmaster views. The script was tested with all the versions I could find (7.31, 9.3, 9.4, 10, 11.1, 11.5 and 11.70) and it worked without issues in all of them. So if you're upgrading an old system and want to make sure you clear any pending inplace altered tables this can be a great help.

The SQL containing the script is at the bottom of this article and I will not dig into it with great detail. The challenges I got in developing it were mainly to understand how the needed data was already present in sysmaster views and also on interpreting this data (the representation is different depending on the "endianess" of your platform. Again, for the first one the help from Andreas Legner was precious and for the second one a special thank you goes to Art Kagel. Both of them helped me to review the script and to fix some nasty bugs I had in my first attempts.

Usage

In order to use this procedure, you'll need to copy the script code below, paste it into dbaccess and run it against any of your instance databases. It will create a function called get_pending_ipa() that will return the following fields:
  • Database
  • Table name
  • Partition name
  • Object type (can be table, partition or partition main)
  • Partition number
  • Partition lockid (the partnum of the main partition for fragmented tables)
  • Table structure version
  • Number of pages remaining in this version
If you need to create this against a version 7 (pre-V9) you should change the header and footer as commented in the script.
To execute just run

execute function get_pending_ipa();
or
execute procedure get_pending_ipa();

Disclaimer

Although the script was tested as much as I could, please understand that is comes with no guarantee. Use at your one risk. Neither me nor my employer can be considered liable for any harm done by it (difficult to happen since it only SELECTs), or more important for bad decisions taken based on it's output. This is just the usual disclaimer. Naturally I've done my best to make sure it works. If you find any error in the script or if you have any suggestion, feel free to contact me.



Versão Portuguesa:

Introdução

O assunto das tabelas com inplace ALTERs (optei por não traduzir o termo) tem estado sempre presente no Informix. Dizemos que uma tabela tem um inplace ALTER quando uma instrução ALTER TABLE deu origem a uma nova definição (schema) de tabela, mas a mesma não foi fisicamente alterada. Isto é muito útil em tabelas grandes e/ou com muitos acessos. Permite que se faça um ALTER TABLE muito rápido (instantâneo) e com um impacto reduzido no consumo de recursos do sistema. Internamente o Informix faz algo muito simples do ponto de vista do utilizador mas que pode ser bastante complexo se visto pelo ângulo do motor: É criada uma nova definição da estrutura da tabela, e a partir desse momento qualquer instrução (DML) que afecte os dados usará essa nova versão. As páginas já existentes da tabela mantêm-se na versão antiga, mas sempre que façamos um SELECT o motor converte a linha(s) para o novo formato. Adicionalmente, se fizermos um UPDATE a página onde estiver guardado o registo(s) será convertida pelo motor para o novo formato.

Por exemplo, se tivermos uma tabela com muitas páginas de dados, e adicionar-mos uma coluna, isto será feito com um inplace ALTER. Mas alguns tipos de ALTER TABLE não permitem um inplace ALTER. Caso mudemos uma coluna de CHAR(5) para SMALLINT, isto será um slow ALTER. Habitualmente isto acontece se o sistema não puder garantir imediatamente que os dados existentes têm representação ou podem ser guardados no novo tipo de dados (no caso anterior o CHAR(5) pode ter caracteres não numéricos). Se o motor decide que pode fazer um inplace ALTER não temos forma de o inibir. Ou seja, a alteração será forçosamente feita com inplace ALTER e se desejarmos forçar a mudança física temos de fazer o que normalmente se designa de dummy UPDATE: UPDATE tabela SET coluna = coluna;

Impactos dos in place ALTERs

Existem vários impactos em ter tabelas com inplace ALTERs. O mais imediato é que os UPDATEs serão ligeiramente mais lentos, pois toda a página tem de ser convertida e escrita. Note-se que isto não implica maior carga de I/O, pois a página é a unidade de escrita mais pequena do motor. Por outras palavras, mesmo que a sua tabela não tenha mais que uma versão da sua definição, quando mudamos uma linha contida numa página, toda a página (pelo menos) será escrita. Mas as outras linhas da mesma página não são alteradas. E mudar de uma versão anterior da definição para a atual pode implicar que nem todas as linhas caibam na página depois de convertidas. Isso sim, pode implicar mais I/O.
Outro potencial problema é que como em qualquer outra área de código do motor, o mecanismo de inplace ALTER pode ter problemas ou erros. Este aspeto é muitas vezes sobrevalorizado, mas é um facto que já tivemos problemas que só aconteciam em tabelas com inplace ALTERs.

Mas o verdadeiro problema habitualmente associado com os inplace ALTERs diz respeito aos upgrades e downgrades. Através das versões tem existido grande controvérsia relativamente às verdadeiras implicações de se efetuarem conversões (ou upgrades inplace) de versão existindo tabelas com inplace ALTERs pendentes (por pendentes quer-se dizer que têm efetivamente páginas com mais que uma versão de definição ou schema da tabela). Procurei pelos guias de migração de várias versões (7.3, 9.2, 9.4, 10, 11.1, 11.5 e 11.7) e são quase absolutamente consistentes: Pode fazer-se o upgrade com inplace ALTERs pendentes mas não se pode fazer o inverso (regressão). A única exceção a esta regra está no manual da versão 9.4 que refere que os mesmos têm de ser removidos antes de se efectuar a conversão.

A razão para permitir conversões, mas não regredir é bastante simples e compreensível: O Informix garante que a versão N+1 consegue lidar com toas as possibilidades de inplace ALTERs da versão N ou anteriores. Mas dado que em cada versão podem ser adicionadas novas situações onde o motor consegue fazer um inplace ALTER, não podemos correr o risco de portar um inplace ALTER pendente para uma versão anterior que não sabe como lidar com ele. Deixe-me lembrar que um inplace ALTER obriga a que o motor consiga mapear os dados de um formato para outro (com mais colunas, ou tipos de dados diferentes etc.)

No momento da escrita deste artigo não consegui verificar se a exceção no guia de migração da versão 9.4 se pode justificar com um erro de documentação ou se tem outro fundamento. Mas o facto é que a maioria dos utilizadores assumem que têm de completar (ou eliminar) os inplace ALTERs pendentes antes das conversões (upgrades). Uma razão válida para este raciocínio é que caso seja necessário regredir para a versão original não se quererá perder tempo a executar os dummy updates. Ou seja, eliminar os inplace ALTERs pendentes, antes da conversão, pode poupar tempo precioso caso se verifique a necessidade de regredir. O manual da versão 11.7 vai um pouco mais longe e refere que só é necessário remover os inplace ALTERs pendentes, se os mesmos foram gerados já na versão 11.7

Qualquer um anterior (que já existisse na versão original) não necessitará de ser eliminado.
Como um aparte, permita-me que diga que a única vez que fiz regressões foi num workshop para parceiros e clientes com o objetivo de demonstrar a funcionalidade. Nunca tive a necessidade de efectuar isto numa situação real em clientes. Em qualquer caso existem várias limitações às regressões que podem constituir verdadeiros desafios, pelo que os inplace ALTERs não deveriam ser a maior preocupação.

Identificar inplace ALTERs pendentes

Agora que vimos em que situações devemos remover os inplace ALTERs pendentes, chegamos a outro tópico que é alvo frequente de perguntas e discussões. A questão é: Como identificamos as tabelas que possuem páginas de dados com várias versões da sua definição? Há várias respostas e regra geral são controvérsias ou mal explicadas o que levanta enormes confusões. Decorreu uma discussão sobre este tema mais uma vez em finais de 2010. Como é hábito foram dadas três respostas para o problema:
  1. Uma forma rápida (baseada em SQL e tabelas SMI) que nos diz as tabelas que sofreram inplace ALTERs mas não permite saber se ainda estão pendentes (existem ainda páginas de dados com formato antigo). Isto significa que a menos que se refaça completamente a tabela, a mera execução dos dummy UPDATEs não impedirá a tabela de voltar a aparecer na lista gerada por este método.
  2. Uma forma lenta, baseada no resultado do oncheck -pT. Isto efetivamente diz-nos quantas páginas de dados existem para cada formato da tabela. Este método permite realmente identificar os inplace ALTERs pendentes.
  3. Uma ferramenta do suporte técnico que procura na metadata das tabelas e pode fornecer a resposta de forma rápida. O único problema é que não está disponível para os utilizadores em geral

E isto foi a minha motivação para efetuar alguma pesquisa sobre este tema. Após alguma troca de informação com o Andreas Legner do suporte técnico da IBM na Alemanha, consegui criar um script SQL que pode reportar as tabelas com inplace ALTERs pendentes. Este script consegue fornecer o número de páginas existentes em cada versão da definição da tabela. Retorna a base de dados, a tabela, a versão(ões) e quantas páginas contém.

O bom deste script é que é rápido (de uns segundos a poucos minutos para bases de dados muito grandes), e mostra a situação actual. Contrariamente à opção 1) acima, depois de fazermos os dummy UPDATEs numa tabela, essa mesma tabela não volta a aparecer no output do script. Apenas um aviso relativamente a isto: O script percorre o que chamamos de partition headers e estes só são escritos em disco durante um checkpoint. Assim, depois de correr os dummy UPDATEs devem forçar-se um checkpoint (ou esperar que ocorra um) antes de correr novamente o script.

O script traduz-se num procedimento SPL e baseia-se em informação disponível nas views da base de dados sysmaster. O script foi testeado em todas as versões que consegui encontrar (7.31, 9.3, 9.4, 10, 11.1, 11.5 e 11.7) e correru em todas sem problemas. Assim, se estiver a fazer uma conversão de um sistema antigo e quiser limpar todos os inplace ALTERs pendentes nessa instância, isto pode ser uma grande ajuda.

O script SQL contendo o procedimento está disponível no final deste artigo e não vou fazer uma explicação exaustiva do mesmo. Os desafios que enfrentei durante o desenvolvimento do procedimento foram principalmente entender se os os dados necessários estavam representados na base de dados sysmaster e também na interpretação desses dados (a representação dos dados é diferente conforme o "endianess" da plataforma). Mais uma vez, na primeira questão a ajuda do Andreas Legner foi preciosa e para a segunda questão tive a ajuda do Art Kagel a quem enviou um sincero agradecimento. Ambos me ajudaram a rever o procedimento e identificaram alguns bugs feios que tinha nas primeiras tentativas.

Utilização


Para usar esta função terá de copiar o código do script que se encontra no final do artigo, colá-lo num dbaccess (ou outra ferramenta) e executá-lo numa das bases de dados da sua instância. O script irá criar uma função chamada get_pending_ipa() que irá retornar os seguintes valores:

  • Nome da base de dados
  • Nome da tabela
  • Nome da partição
  • Tipo de objecto (pode ser table, partition ou partition main)
  • Número da partição
  • lockid da partição (o número da partição principal para tabelas fragmentadas)
  • Versão da estrutura da tabela
  • Número de páginas ainda existentes nesta versão
Se necessitar de criar a função numa versão 7 (pre-V9) deve alterar o cabeçalho e o final da função de acordo com o recomendado/comentado no código
Para executar basta dar a instrução:

execute function get_pending_ipa();
ou

execute procedure get_pending_ipa();

Exclusão de garantia

Apesar de o script ter sido testado tanto quanto pude, por favor assuma que o mesmo não é fornecido com qualquer tipo de garantia. Utilize-o por sua conta e risco. Nem eu nem o meu empregador poderão ser considerados responsáveis por qualquer mal ou prejuízo derivado do seu uso (difícil dado que apenas faz SELECTs), ou mais importante, por más decisões baseadas no seu output. Isto é apenas o normal termo de des-responsabilização. Naturalmente fiz o meu melhor para assegurar que o procedimento funciona bem e retorna resultados corretos. Qualquer problema que identifique no script ou sugestão de melhoria por favor contacte-me.





SQL script:



CREATE FUNCTION get_pending_ipa() RETURNING
        VARCHAR(128) as database, VARCHAR(128) as table, VARCHAR(128) as partition, VARCHAR(9) as obj_type,
        INTEGER as partnum, INTEGER as lockid, SMALLINT as version, INTEGER as npages
-- For version 7.x use this header instead:
--CREATE PROCEDURE get_pending_ipa() RETURNING VARCHAR(128), VARCHAR(128), VARCHAR(128), VARCHAR(9), INTEGER, INTEGER, SMALLINT, INTEGER;
-- Name: $RCSfile: get_pending_ipa.sql,v $
-- CVS file: $Source: /usr/local/cvs/stable/informix/queries/get_pending_ipa.sql,v $
-- CVS id: $Header: /usr/local/cvs/stable/informix/queries/get_pending_ipa.sql,v 1.5 2011/09/09 20:57:31 fnunes Exp $
-- Revision: $Revision: 1.5 $
-- Revised on: $Date: 2011/09/09 20:57:31 $
-- Revised by: $Author: fnunes $
-- Support: Fernando Nunes - domusonline@gmail.com
-- Licence: This script is licensed as GPL ( http://www.gnu.org/licenses/old-licenses/lgpl-2.0.html )
-- Variables holding the database,tabnames and partnum
DEFINE v_dbsname, v_old_dbsname LIKE sysmaster:systabnames.dbsname;
DEFINE v_tabname, v_partname, v_old_tabname LIKE sysmaster:systabnames.tabname;
DEFINE v_partnum, v_old_partnum LIKE sysmaster:syspaghdr.pg_partnum;
DEFINE v_lockid, v_old_lockid LIKE sysmaster:sysptnhdr.lockid;
DEFINE v_pg_next INTEGER;
DEFINE v_pg_partnum INTEGER;
DEFINE v_obj_type VARCHAR(9);
-- Variables holding the various table versions and respective number of pages pending to migrate
DEFINE v_version SMALLINT;
DEFINE v_pages INTEGER;
-- Hexadecimal representation of version and pending number of pages
DEFINE v_char_version CHAR(6);
DEFINE v_char_pages CHAR(10);
DEFINE v_aux_char CHAR(8);
-- Hexadecimal representation of the slot 6 data. Each 16 bytes will appear as a record that needs to be concatenated
DEFINE v_hexdata VARCHAR(128);
-- Variable to hold the sysmaster:syssltdat hexadecimal representation of each 16 bytes of the slot data
DEFINE v_slot_hexdata CHAR(40);
DEFINE v_aux VARCHAR(128);
DEFINE v_endian CHAR(6);
DEFINE v_offset SMALLINT;
DEFINE v_slotoff SMALLINT;
DEFINE v_dummy INTEGER;
-- In case we need to trace the function... Uncomment the following two lines
--SET DEBUG FILE TO "/tmp/get_pending_ipa.dbg";
--TRACE ON;
-- Now lets find out the Endianess ( http://en.wikipedia.org/wiki/Endianness ) of this platform
-- The data in sysmaster:syssltdat will be different because of possible byte swap
-- Read the first slot of the rootdbs TBLSpace tblspace (0x00100001)
-- The first 4 bytes hold the partition number (0x00100001)
SELECT
        s.hexdata[1,8]
INTO
        v_hexdata
FROM
        sysmaster:syssltdat s
WHERE
        s.partnum = '0x100001' AND
        s.pagenum = 1 AND
        s.slotnum = 1 AND
        s.slotoff = 0;
IF v_hexdata = '01001000'
THEN
        -- Byte swap order, so we're little Endian (Intel, Tru64....)
        LET v_endian = 'LITTLE';
ELSE
        IF v_hexdata = '00100001'
        THEN
                -- Just as we write it (no byte swap), so we're big Endian (Sparc, Power, Itanium...)
                LET v_endian = 'BIG';
        ELSE
                -- Just in case something weird (like a bug(!) or physical modification) happened
                RAISE EXCEPTION -746, 0, 'Invalid Endianess calculation... Check procedure code!!!';
        END IF
END IF
-- Flags to mark the beginning
LET v_hexdata = "-";
LET v_old_dbsname = "-";
LET v_old_tabname = "-";
-- The information we want for each version description will occupy this number of characters
-- in the sysmaster:syssltdat.hexdata notation (after removing spaces). The size depends on the engine version.
LET v_offset=DBINFO('version','major');
IF v_offset >= 10
THEN
        LET v_offset = 48;
ELSE
        LET v_offset = 40;
END IF
LET v_old_lockid = -1;
FOREACH
        -- This query will browse through all the instance partitions, excluding sysmaster database, and will look for
        -- any extended partition header (where partition header "next" field is not 0)
        -- the ABS(...) is just a trick to make partnums that are equal to lock id appear at the end
        SELECT
                t.dbsname, t.tabname, t1.tabname, t.partnum, p.pg_partnum, p.pg_next, h.lockid, ABS(h.lockid - h.partnum)
        INTO
                v_dbsname, v_partname ,v_tabname, v_partnum, v_pg_partnum, v_pg_next, v_lockid, v_dummy
        FROM
                sysmaster:systabnames t,
                sysmaster:syspaghdr p,
                sysmaster:sysptnhdr h,
                sysmaster:systabnames t1
        WHERE
                p.pg_partnum = sysmaster:partaddr(sysmaster:partdbsnum(t.partnum),1) AND
                p.pg_pagenum = sysmaster:partpagenum(t.partnum) AND
                t.dbsname NOT IN ('sysmaster') AND
                h.partnum = t.partnum AND
                t1.partnum = h.lockid AND
                p.pg_next != 0
        ORDER BY
                t.dbsname, t.tabname, 8 DESC, t.partnum
        IF v_lockid = v_partnum
        THEN
                IF v_lockid = v_old_lockid
                THEN
                        LET v_obj_type = "Part Main";
                ELSE
                        LET v_obj_type = "Table";
                END IF
        ELSE
                LET v_obj_type = "Part";
        END IF
      
        LET v_old_lockid = v_lockid;
        WHILE v_pg_next != 0
                -- Find if we're dealing with a fragmented table or not...
                -- While this extended partition page points to another one...
                -- Get all the slot 6 data (where the version metadata is stored - version, number of pages, descriptor page etc.
                FOREACH
                SELECT
                        REPLACE(s.hexdata, ' '), s.slotoff, p.pg_next
                INTO
                        v_slot_hexdata, v_slotoff, v_pg_next
                FROM
                        sysmaster:syspaghdr p,
                        sysmaster:syssltdat s
                WHERE
                        s.partnum = p.pg_partnum AND
                        s.pagenum = p.pg_pagenum AND
                        s.slotnum = 6 AND
                        p.pg_partnum = v_pg_partnum AND
                        p.pg_pagenum = v_pg_next
                IF ( v_dbsname != v_old_dbsname OR v_tabname != v_old_tabname OR v_partnum != v_old_partnum)
                THEN
                        LET v_old_dbsname = v_dbsname;
                        LET v_old_tabname = v_tabname;
                        LET v_old_partnum = v_partnum;
                        -- First iteraction for each table
                        LET v_hexdata = v_slot_hexdata;
                ELSE
                        -- Next iteractions for each table
                        LET v_hexdata = TRIM(v_hexdata) || v_slot_hexdata;
                        IF LENGTH(v_hexdata) >= v_offset
                        THEN
                                -- We already have enough data for a version within a table
                                -- Note that we probably have part of the next version description in v_hexdata
                                -- So we need to copy part of it, and keep the rest for next iteractions
                                LET v_aux=v_hexdata;
                                LET v_hexdata=SUBSTR(v_aux,v_offset+1,LENGTH(v_aux)-v_offset);
                      
                                -- Split the version and number of pending pages part...
                                LET v_char_version = v_aux[1,4];
                                LET v_char_pages = v_aux[9,16];
                                -- Create a usable hex number. Prefix it with '0x' and convert due to little endian if that's the case
                                IF v_endian = "BIG"
                                THEN
                                        LET v_char_version = '0x'||v_char_version;
                                        LET v_char_pages = '0x'||v_char_pages;
                                ELSE
                                        LET v_aux_char = v_char_version;
                                        LET v_char_version[5]=v_aux_char[1];
                                        LET v_char_version[6]=v_aux_char[2];
                                        LET v_char_version[4]=v_aux_char[4];
                                        LET v_char_version[3]=v_aux_char[3];
                                        LET v_char_version[2]='x';
                                        LET v_char_version[1]='0';
                                        LET v_aux_char = v_char_pages;
                                        LET v_char_pages[9]=v_aux_char[1];
                                        LET v_char_pages[10]=v_aux_char[2];
                                        LET v_char_pages[7]=v_aux_char[3];
                                        LET v_char_pages[8]=v_aux_char[4];
                                        LET v_char_pages[6]=v_aux_char[6];
                                        LET v_char_pages[5]=v_aux_char[5];
                                        LET v_char_pages[3]=v_aux_char[7];
                                        LET v_char_pages[4]=v_aux_char[8];
                                        LET v_char_pages[2]='x';
                                        LET v_char_pages[1]='0';
                                END IF
                                -- HEX into DEC (integer)
                                LET v_version = TRUNC(v_char_version + 0);
                                LET v_pages = TRUNC(v_char_pages + 0);
                                IF v_pages > 0
                                THEN
                                        -- This version has pending pages so show it...
                                        RETURN TRIM(v_dbsname), TRIM(v_tabname), TRIM(v_partname), TRIM(v_obj_type), v_partnum, v_lockid, v_version, v_pages WITH RESUME;
                                END IF
                        END IF
                END IF
                END FOREACH
                IF LENGTH(v_hexdata) >= v_offset
                THEN
                        -- If we still have data to process...
                        LET v_aux=v_hexdata;
      
                        LET v_char_version = v_aux[1,4];
                        LET v_char_pages = v_aux[9,16];
                        IF v_endian = "BIG"
                        THEN
                                LET v_char_version = '0x'||v_char_version;
                                LET v_char_pages = '0x'||v_char_pages;
                        ELSE
                                LET v_aux_char = v_char_version;
                                LET v_char_version[5]=v_aux_char[1];
                                LET v_char_version[6]=v_aux_char[2];
                                LET v_char_version[4]=v_aux_char[4];
                                LET v_char_version[3]=v_aux_char[3];
                                LET v_char_version[2]='x';
                                LET v_char_version[1]='0';
                                LET v_aux_char = v_char_pages;
                                LET v_char_pages[9]=v_aux_char[1];
                                LET v_char_pages[10]=v_aux_char[2];
                                LET v_char_pages[7]=v_aux_char[3];
                                LET v_char_pages[8]=v_aux_char[4];
                                LET v_char_pages[6]=v_aux_char[6];
                                LET v_char_pages[5]=v_aux_char[5];
                                LET v_char_pages[3]=v_aux_char[7];
                                LET v_char_pages[4]=v_aux_char[8];
                                LET v_char_pages[2]='x';
                                LET v_char_pages[1]='0';
                        END IF
                        -- HEX into DEC (integer)
                        LET v_version = TRUNC(v_char_version + 0);
                        LET v_pages = TRUNC(v_char_pages + 0);
                        IF v_pages > 0
                        THEN
                                -- This version has pending pages so show it...
                                RETURN TRIM(v_dbsname), TRIM(v_tabname), TRIM(v_partname), TRIM(v_obj_type), v_partnum, v_lockid, v_version, v_pages WITH RESUME;
                        END IF
                END IF
        END WHILE
END FOREACH;
END FUNCTION;
-- For version 7.x use this close statement instead:
--END PROCEDURE;