Permanently delete recodes from a table
execute procedure sp_zaptable
('T:\studies\82289\data_extract\extract.dbf')
http://devzone.advantagedatabase.com/dz/webhelp/advantage8.1/supported_statements/miscellaneous_functions.htm
Select column, data type and status from a table
select name, field_type, field_can_be_null from System.columns
where parent = 'rec_sample_test'
order by 2,1
Merge tables
MERGE web_sample_test p1 USING "//loki/web/secure/82124/cust/US/live/rec/study.add".sample p2
ON ( p1.sam_no = p2.sam_no and p1.waveno =p2.waveno)
WHEN MATCHED THEN
UPDATE SET p1.surname = p2.surname, p1.forename = p2.forename
WHEN NOT MATCHED THEN
INSERT (sam_no, waveno, forename, surname) VALUES (p2.sam_no, p2.waveno, p2.forename, p2.surname)
Advantage database server index
http://devzone.advantagedatabase.com/dz/webhelp/Advantage10.1/advantage_kwindex_static.html
Monday, March 26, 2012
PostgreSQL 9.2 Volume 2 Part ii
Chapter 11. Indexes
If there are many rows a tabl and only a few rows (perhaps zero or one) that would be returned by a query, this is clearly an inefficient method. But if the system has been instructed to maintain an index on the id column, it can use a more efficient method for locating matching rows. For instance, it might only have to walk a few levels deep into a search tree.
CREATE INDEX test1_id_index ON test1 (id);
B-trees indexes can handle equality and range queries on data.involved in a comparison using one of these operators: <, <=, =, >=, >, BETWEEN, IN, IS NULL, IS NOT NULL and patter matching (LIKE). B-tree indexes can also be used to retrieve data in sorted order. This is not always faster than a simple scan and sort, but it is often helpful.
B-tree indexes can also be used to retrieve data in sorted order. This is not always faster than a simple scan and sort, but it is often helpful. Hash indexes can only handle simple equality comparisons. Involved in a comparison using the = operator.
CREATE INDEX name ON table USING hash (column);
Multi column Indexes CREATE INDEX test2_mm_idx ON test2 (major, minor);
http://www.postgresql.org/docs/9.1/interactive/indexes.html
Chapter 12. Full Text Search
http://www.postgresql.org/docs/9.1/interactive/index.html
Transaction Isolation Levels
http://www.postgresql.org/docs/9.1/interactive/index.html
Chapter 13. Concurrency Control
data consistency is maintained by using a multiversion model (Multiversion Concurrency Control, MVCC). This means that while querying a database each transaction sees a snapshot of data as it was some time ago, regardless of the current state of the underlying data. This protects the transaction from viewing inconsistent data that could be caused by (other) concurrent transaction updates on the same data rows, providing transaction isolation for each database session.
MVCC model of concurrency control rather than locking is that in MVCC locks acquired for querying (reading) data do not conflict with locks acquired for writing data which guarantee even when providing the strictest level of transaction isolation through the use of an innovative Serializable Snapshot Isolation (SSI) level.
13.2. Transaction Isolation
The SQL standard defines four levels of transaction isolation. The most strict is Serializable.The phenomena which are prohibited are various levels are:
Dirty read - A transaction reads data written by a concurrent uncommitted transaction.
Non-repeatable read - A transaction re-reads data it has previously read and finds that data has been modified by another transaction (that committed since the initial read).
phantom read - A transaction re-executes a query returning a set of rows that satisfy a search condition and finds that the set of rows satisfying the condition has changed due to another recently-committed transaction.
The four transaction isolation levels and the corresponding behaviors are described in Table 13-1.Dirty read - A transaction reads data written by a concurrent uncommitted transaction.
Non-repeatable read - A transaction re-reads data it has previously read and finds that data has been modified by another transaction (that committed since the initial read).
phantom read - A transaction re-executes a query returning a set of rows that satisfy a search condition and finds that the set of rows satisfying the condition has changed due to another recently-committed transaction.
Transaction Isolation Levels
| Isolation Level | Dirty Read | Nonrepeatable Read | Phantom Read |
|---|---|---|---|
| Read uncommitted | Possible | Possible | Possible |
| Read committed | Not possible | Possible | Possible |
| Repeatable read | Not possible | Not possible | Possible |
| Serializable | Not possible | Not possible | Not possible |
13.3. Explicit Locking
http://www.postgresql.org/docs/9.1/interactive/explicit-locking.html
Chapter 14. Performance Tips
http://www.postgresql.org/docs/9.1/interactive/explicit-locking.html
Chapter 14. Performance Tips
14.1. Using EXPLAIN
PostgreSQL devises a query plan for each query it receives. You can use the EXPLAIN command to see what query plan the planner creates for any query.
The structure of a query plan is a tree of plan nodes. The first line (topmost node) has the estimated total execution cost for the plan; it is this number that the planner seeks to minimize.
EXPLAIN SELECT * FROM tenk1;
QUERY PLAN
-------------------------------------------------------------
Seq Scan on tenk1 (cost=0.00..458.00 rows=10000 width=244)
The numbers that are quoted by EXPLAIN are (left to right):
- Estimated start-up cost (time expended before the output scan can start, e.g., time to do the sorting in a sort node)
- Estimated total cost (if all rows are retrieved, though they might not be; e.g., a query with a LIMIT clause will stop short of paying the total cost of the Limit plan node's input node)
- Estimated number of rows output by this plan node (again, only if executed to completion)
- Estimated average width (in bytes) of rows output by this plan node
14.4. Populating a Database
One might need to insert a large amount of data when first populating a database.
** Disable Auto commit
When using multiple INSERTs, turn off autocommit and just do one commit at the end. An additional benefit of doing all insertions in one transaction is that if the insertion of one row were to fail then the insertion of all rows inserted up to that point would be rolled back, so you won't be stuck with partially loaded data.
** Use COPY
Use COPY to load all the rows in one command, instead of using a series of INSERT commands. The COPY command is optimized for loading large numbers of rows; it is less flexible than INSERT, but incurs significantly less overhead for large data loads. Since COPY is a single command, there is no need to disable autocommit if you use this method to populate a table.
COPY is fastest when used within the same transaction as an earlier CREATE TABLE or TRUNCATE command. In such cases no WAL needs to be written, because in case of an error, the files containing the newly loaded data will be removed anyway. However, this consideration only applies when wal_level is minimal as all commands must write WAL otherwise.
** Remove Indexes
If you are loading a freshly created table, the fastest method is to create the table, bulk load the table's data using COPY, then create any indexes needed for the table. Creating an index on pre-existing data is quicker than updating it incrementally as each row is loaded.
If you are adding large amounts of data to an existing table, it might be a win to drop the indexes, load the table, and then recreate the indexes. Of course, the database performance for other users might suffer during the time the indexes are missing. One should also think twice before dropping a unique index, since the error checking afforded by the unique constraint will be lost while the index is missing.
** Remove Foreign Key Constraints
Just as with indexes, a foreign key constraint can be checked "in bulk" more efficiently than row-by-row. So it might be useful to drop foreign key constraints, load data, and re-create the constraints. Again.
when you load data into a table with existing foreign key constraints, each new row requires an entry in the server's list of pending trigger events (since it is the firing of a trigger that checks the row's foreign key constraint). Loading many millions of rows can cause the trigger event queue to overflow available memory, leading to intolerable swapping or even outright failure of the command.Alternative method is to split up the load operation into smaller transactions
** Increase maintenance_work_mem
Temporarily increasing the maintenance_work_mem configuration variable. This will help to speed up CREATE INDEX commands and ALTER TABLE ADD FOREIGN KEY commands. It won't do much for COPY itself, so this advice is only useful when you are using one or both of the above techniques.
** Increase checkpoint_segments
Temporarily increasing the "checkpoint_segments" configuration variable. This is because loading a large amount of data into PostgreSQL will cause checkpoints to occur more often than the normal checkpoint frequency (specified by the checkpoint_timeout configuration variable). Whenever a checkpoint occurs, all dirty pages must be flushed to disk. By increasing checkpoint_segments temporarily during bulk data loads, the number of checkpoints that are required can be reduced.
** Run ANALYZE Afterwards
Whenever you have significantly altered the distribution of data within a table, running ANALYZE is strongly recommended. which ensures that the planner has up-to-date statistics about the table. With no statistics or obsolete statistics, the planner might make poor decisions during query planning. Note that if the autovacuum daemon is enabled, it might run ANALYZE automatically
Friday, February 17, 2012
Regular Expressions
Regular Expressions
. (dot) Any single character except newline
* zero or more occurances of any character
[...] Any single character specified in the set
[^...] Any single character not specified in the set
^ Anchor - beginning of the line
$ Anchor - end of line
\< Anchor - begining of word
\> Anchor - end of word
\(...\) Grouping - usually used to group conditions
\n Contents of nth grouping
[...] Set Examples
[A-Z] The SET from Capital A to Capital Z
[a-z] The SET from lowercase a to lowercase z
[0-9] The SET from 0 to 9 (All numerals)
[./=+] The SET containing . (dot), / (slash), =, and +
[-A-F] The SET from Capital A to Capital F and the dash (dashes must be specified first)
[0-9 A-Z] The SET containing all capital letters and digits and a space
[A-Z][a-zA-Z] In the first position, the SET from Capital A to Capital Z
Regular Expression Examples
/Hello/ Matches if the line contains the value Hello
/^TEST$/ Matches if the line contains TEST by itself
/^[a-zA-Z]/ Matches if the line starts with any letter
^.*SET.*$ Select whole line which start with "SET"
/^[a-z].*/ Matches if the first character of the line is a-z and there is at least one more of any character
"^.* as " select a phase from beginning up to as word. Can be used to replace "xxxx as c_q6" as "c_q6"
following it
/2134$/ Matches if line ends with 2134
/\(21|35\)/ Matches is the line contains 21 or 35
/[0-9]*/ Matches if there are zero or more numbers in the line
/^[^#]/ Matches if the first character is not a # in the line
\r\n[^\r\nO] Matches the lines start with \r\n in previous line but not start with O
Notes:
1. Regular expressions are case sensitive
2. Regular expressions are to be used where pattern is specified
. (dot) Any single character except newline
* zero or more occurances of any character
[...] Any single character specified in the set
[^...] Any single character not specified in the set
^ Anchor - beginning of the line
$ Anchor - end of line
\< Anchor - begining of word
\> Anchor - end of word
\(...\) Grouping - usually used to group conditions
\n Contents of nth grouping
[...] Set Examples
[A-Z] The SET from Capital A to Capital Z
[a-z] The SET from lowercase a to lowercase z
[0-9] The SET from 0 to 9 (All numerals)
[./=+] The SET containing . (dot), / (slash), =, and +
[-A-F] The SET from Capital A to Capital F and the dash (dashes must be specified first)
[0-9 A-Z] The SET containing all capital letters and digits and a space
[A-Z][a-zA-Z] In the first position, the SET from Capital A to Capital Z
Regular Expression Examples
/Hello/ Matches if the line contains the value Hello
/^TEST$/ Matches if the line contains TEST by itself
/^[a-zA-Z]/ Matches if the line starts with any letter
^.*SET.*$ Select whole line which start with "SET"
/^[a-z].*/ Matches if the first character of the line is a-z and there is at least one more of any character
"^.* as " select a phase from beginning up to as word. Can be used to replace "xxxx as c_q6" as "c_q6"
following it
/2134$/ Matches if line ends with 2134
/\(21|35\)/ Matches is the line contains 21 or 35
/[0-9]*/ Matches if there are zero or more numbers in the line
/^[^#]/ Matches if the first character is not a # in the line
\r\n[^\r\nO] Matches the lines start with \r\n in previous line but not start with O
Notes:
1. Regular expressions are case sensitive
2. Regular expressions are to be used where pattern is specified
Labels:
Unix
Tuesday, February 14, 2012
Bash command with parameters to run PDI transformation
fatigue_export.sh file
Define the parameters name in transformation properties and keep the default values empty
windows batch file with few validations
PDI Transformation properties (Ctrl + T)#!/bin/bashecho "Enter the name of the sample file(eg CVAR_Good_20110104):"read sample_file #sample_file is the variable which pick the entered valueecho "Enter the wave (eg 1):" read wave_no #wave_no is the variable/home/kettle4/pan.sh -file=/home/pdi_transforms/ibmcvar/fatigue/pdi_transforms/fatigue_export.ktr -norep -param:FILENAME=$sample_file -param:WAVE=$wave_no -XX:-UseGCOverheadLimit
Define the parameters name in transformation properties and keep the default values empty
windows batch file with few validations
@ECHO OFF ECHO Enter your job number %1 SET /P userin= IF "%userin%"=="" ( ECHO ---------------------------------- echo "You should enter a job number..." ECHO ---------------------------------- pause exit ) IF NOT EXIST Q:\secure\%userin% ( ECHO %userin% Job directory doesn't exist in Q:\secure\ ECHO ----------------------------- ECHO Please enter valid job number ECHO ----------------------------- pause exit ) IF EXIST Q:\secure\%userin% ( ECHO ------------------------------------------------- ECHO Maximum sample numbers are being calculated...... ECHO This may take few minutes. please wait........... L:\kettle4.2\Kitchen.bat /file="W:\Panelsample_iss\max_sample_no\max_sample_no.kjb" "-param:jobno=%userin%" > nul ECHO ------------------------------------------------- ECHO ------------------------------------------------- ECHO Please check max_sample_nos_%userin% file inside IF EXIST w:\%userin%\Sample\ToLoad ( echo w:\%userin%\Sample\ToLoad pause ) else ( echo w:\%userin%US\Sample\ToLoad directory pause ) )
Labels:
Pentaho Data Integration,
Unix
Monday, January 16, 2012
Appendix F. Additional Supplied Modules
F.15. fuzzystrmatch
F.15.1. Soundex
The Soundex system is a method of matching similar-sounding names by converting them to the same code. (Soundex is not very useful for non-English names). The soundex function converts a string to its Soundex code. The difference function converts two strings to their Soundex codes and then reports the number of matching code positions. Since Soundex codes have four characters, the result ranges from zero to four, with zero being no match and four being an exact match. (Thus, the function is misnamed — similarity would have been a better name.)
SELECT soundex('Anne'), soundex('Ann'), difference('Anne', 'Ann');
SELECT soundex('Anne'), soundex('Andrew'), difference('Anne', 'Andrew');
SELECT soundex('Anne'), soundex('Margaret'), difference('Anne', 'Margaret');
F.15.3. Metaphone
Metaphone & Double Metaphone are algorithms that produce variable length keys for indexing words by their sound. definitely better compared to SOUNDEX because it is more precise compared to the fixed 4 character code of SOUNDEX.
Metaphone, like Soundex, is based on the idea of constructing a representative code for an input string. Two strings are then deemed similar if they have the same codes.(source has to be a non-null string with a maximum of 255 characters)
SELECT metaphone('GUMBO', 4);
metaphone
-----------
KM
F.15.4. Double Metaphone
The Double Metaphone system computes two "sounds like" strings for a given input string — a "primary" and an "alternate". In most cases they are the same, but for non-English names especially they can be a bit different, depending on pronunciation. These functions compute the primary and alternate codes:
The Double Metaphone system computes two "sounds like" strings for a given input string — a "primary" and an "alternate". In most cases they are the same, but for non-English names especially they can be a bit different, depending on pronunciation. These functions compute the primary and alternate codes:
select dmetaphone('gumbo');
dmetaphone
------------
KMP
Labels:
Postgres
Pass Cursor data through a function
ParentFunction - pass cursor data through function
CREATE OR REPLACE FUNCTION process_original_matched_records()
RETURNS void AS
$BODY$
DECLARE
datarec to_process_struct%ROWTYPE;
datarec1 to_process_struct;
BEGIN
perform select_records_to_process();
for datarec in select * from to_process where original_check=1 and puin is not NULL and update_status is NULL LOOP
--RAISE NOTICE 'in process_orginal_matched_records before upsert % ', datarec.puin;
datarec1 := datarec;
--RAISE NOTICE 'in process_orginal_matched_records after datarec1=datarec % ', datarec.puin;
PERFORM upsert_all_tables(datarec, cast(1 as smallint), datarec.puin, cast(1 as smallint));
END LOOP;
END;
$BODY$
LANGUAGE plpgsql VOLATILE;
Child Function - use cursor date from parent table
CREATE OR REPLACE FUNCTION upsert_all_tables(datarec to_process_struct, i_update_status smallint, puin_to_use character, i_statusid smallint)
RETURNS void AS
$BODY$
DECLARE
voicephone text; orig_statusid smallint; update_statusid smallint;verified smallint;
BEGIN
insert into pre_update_data
select ...... m.prim_specialityid, m.sub_specialityid,m.hp_id,c.panelid,now() as update_date, datarec.studyid, datarec.sam_no, i.biz1,i.biz2,i.biz3,i.biz4,i.biz5,i.biz6
from contact c inner join address a on c.panelid=a.panelid left outer join combined_phones cp on cp.panelid=c.panelid
left outer join medical_extra m on m.panelid=c.panelid left outer join it_demogs i on i.panelid=c.panelid
where c.panelid=puin_to_use;
if datarec.stat_type in ('WN','DS','FX') THEN voicephone := NULL;
ELSE
voicephone := datarec.voicephone;
END IF;
if datarec.studyid in (select studyid from study inner join survey on study.surveyid=survey.surveyid
inner join project on project.projectid=survey.projectid where project.projectid=424) THEN
if datarec.stat_type='CP' THEN
verified := 1;
ELSE
verified := 0;
END IF;
END IF;
PERFORM upsert_phone_or_email(puin_to_use, datarec.fax, 5, 'phone');
PERFORM upsert_phone_or_email(puin_to_use, datarec.deptphone, 6, 'phone');
select statusid from contact into orig_statusid where panelid=puin_to_use;
IF FOUND THEN
select assign_statusid(datarec.stat_type, orig_statusid) into update_statusid;
ELSE
update_statusid := i_statusid;
END IF;
PERFORM * from webinc_track where puin=puin_to_use;
IF FOUND THEN
RAISE NOTICE 'Here is a puin in upsert_all_tables before block 1 %', datarec.puin;
PERFORM upsert_contact_details(puin_to_use, datarec.salutation, datarec.forename, datarec.surname, datarec.title, cast(datarec.pxtype as integer), orig_statusid, verified);
RAISE NOTICE 'Here is a puin in upsert_all_tables after block 1 %', datarec.puin;
END IF;
PERFORM upsert_address_details(puin_to_use, datarec.company, datarec.street, datarec.city, datarec.state, datarec.zipcode, cast(datarec.ccode as smallint));
END IF;
if datarec.pxtype = 1 THEN
PERFORM upsert_it_demogs(
puin_to_use,
cast(datarec.biz1 as integer),
END IF;
PERFORM upsert_contact_statistics(puin_to_use, datarec.stat_type);
update interviewed_sample set update_status=i_update_status, update_date=current_date, panelmatchpuin=puin_to_use where studyid=datarec.studyid and sam_no=datarec.sam_no;
END;
$BODY$
LANGUAGE plpgsql VOLATILE
Labels:
Postgres
Thursday, December 22, 2011
Useful PostgreSQL Commands
Update statement joining tables
update deduction set sum=sum+1 from employee e
where e.id=deduction.employeeid and e.groupid=2;
update hall_sample h set panel_id = s.panelid
from (select lower(trim(address)) as email, e.panelid from email e
inner join (select panelid from contact where statusid = 1 and typeid = 2) AS c
on e.panelid = c.panelid) AS s where lower(trim(h.email)) = lower(trim(s.email))
Find duplicates when more than one unique values
select s.studyid, s.sam_no, count(*) from table_name i
inner join table_name s on (i.studyid = s.studyid AND i.sam_no = s.sam_no)
group by 1,2 having count(*) > 1
SELECT key, count(*) FROM ibmcvar.disposition GROUP BY 1 HAVING count(*) > 1;
SELECT * FROM ibmcvar.disposition where key in (select key from (SELECT key, count(*)
FROM ibmcvar.disposition
GROUP BY 1 HAVING count(*) > 1) as t) order by key
Delete one from duplicate rows (minimum fatiguedate)
delete from ibmcvar_2012.disposition where (key, fatiguedate) in (
select key, min(fatiguedate) from ibmcvar_2012.disposition where key in
(select key from (select key, count(*) from ibmcvar_2012.disposition
group by 1 having count(*) > 1 ) as x )
group by 1 order by 2 )
Find records with latest update value
SELECT * FROM table a JOIN (SELECT ID, max(date) maxDate
FROM table GROUP BY ID) b ON a.ID = b.ID AND a.date = b.maxDate
Add sequence to existing column in existing table
First set the maximum number of the field as current value in sequence'
ALTER TABLE ccadmin.tblemployees ALTER COLUMN recordid SET DEFAULT nextval('ccadmin.tblemployees_sec'::regclass)
Select last 5 values for 2 columns as fields for selected column value(person, project)
select puin,
max(case when (row_number = 1) then job_number else null end ) as jobnumber1,
max(case when (row_number = 1) then web_stat_type else null end ) as stat_type1,
max(case when (row_number = 2) then job_number else null end ) as jobnumber2,
max(case when (row_number = 2) then web_stat_type else null end ) as stat_type2,
max(case when (row_number = 3) then job_number else null end ) as jobnumber3,
max(case when (row_number = 3) then web_stat_type else null end ) as stat_type3,
max(case when (row_number = 4) then job_number else null end ) as jobnumber4,
max(case when (row_number = 4) then web_stat_type else null end ) as stat_type4,
max(case when (row_number = 5) then job_number else null end ) as jobnumber5,
max(case when (row_number = 5) then web_stat_type else null end ) as stat_type5
from ( WITH summary AS (
select puin, called_or_mailed_date, job_number, stat_type as web_stat_type,
row_number() over (PARTITION BY puin order by called_or_mailed_date desc)
from reports.add_fields2_samples
where puin in ('C3CE780B-0E3F-4A0E-BA3A', 'EF9ADA63-2D5F-427C-989E')
group by 1,2,3,4 order by puin asc, called_or_mailed_date desc )
SELECT * FROM summary where row_number < 6 ) as a
group by 1

update deduction set sum=sum+1 from employee e
where e.id=deduction.employeeid and e.groupid=2;
update hall_sample h set panel_id = s.panelid
from (select lower(trim(address)) as email, e.panelid from email e
inner join (select panelid from contact where statusid = 1 and typeid = 2) AS c
on e.panelid = c.panelid) AS s where lower(trim(h.email)) = lower(trim(s.email))
Find duplicates when more than one unique values
select s.studyid, s.sam_no, count(*) from table_name i
inner join table_name s on (i.studyid = s.studyid AND i.sam_no = s.sam_no)
group by 1,2 having count(*) > 1
SELECT key, count(*) FROM ibmcvar.disposition GROUP BY 1 HAVING count(*) > 1;
SELECT * FROM ibmcvar.disposition where key in (select key from (SELECT key, count(*)
FROM ibmcvar.disposition
GROUP BY 1 HAVING count(*) > 1) as t) order by key
Delete one from duplicate rows (minimum fatiguedate)
delete from ibmcvar_2012.disposition where (key, fatiguedate) in (
select key, min(fatiguedate) from ibmcvar_2012.disposition where key in
(select key from (select key, count(*) from ibmcvar_2012.disposition
group by 1 having count(*) > 1 ) as x )
group by 1 order by 2 )
Find records with latest update value
SELECT * FROM table a JOIN (SELECT ID, max(date) maxDate
FROM table GROUP BY ID) b ON a.ID = b.ID AND a.date = b.maxDate
Add sequence to existing column in existing table
First set the maximum number of the field as current value in sequence'
ALTER TABLE ccadmin.tblemployees ALTER COLUMN recordid SET DEFAULT nextval('ccadmin.tblemployees_sec'::regclass)
Select last 5 values for 2 columns as fields for selected column value(person, project)
select puin,
max(case when (row_number = 1) then job_number else null end ) as jobnumber1,
max(case when (row_number = 1) then web_stat_type else null end ) as stat_type1,
max(case when (row_number = 2) then job_number else null end ) as jobnumber2,
max(case when (row_number = 2) then web_stat_type else null end ) as stat_type2,
max(case when (row_number = 3) then job_number else null end ) as jobnumber3,
max(case when (row_number = 3) then web_stat_type else null end ) as stat_type3,
max(case when (row_number = 4) then job_number else null end ) as jobnumber4,
max(case when (row_number = 4) then web_stat_type else null end ) as stat_type4,
max(case when (row_number = 5) then job_number else null end ) as jobnumber5,
max(case when (row_number = 5) then web_stat_type else null end ) as stat_type5
from ( WITH summary AS (
select puin, called_or_mailed_date, job_number, stat_type as web_stat_type,
row_number() over (PARTITION BY puin order by called_or_mailed_date desc)
from reports.add_fields2_samples
where puin in ('C3CE780B-0E3F-4A0E-BA3A', 'EF9ADA63-2D5F-427C-989E')
group by 1,2,3,4 order by puin asc, called_or_mailed_date desc )
SELECT * FROM summary where row_number < 6 ) as a
group by 1
Labels:
Postgres
Monday, November 21, 2011
PostgreSQL 9.2 Volume 2 - Summary
4. SQL Syntax
Value Expression
http://developer.postgresql.org/pgdocs/postgres/sql-expressions.html
Calling Function
Named notation is especially useful for functions that have a large number of parameters (makes the associations between parameters and actual arguments more explicit and reliable)
Positional notation, a function call is written with its argument values in the same order as they are defined in the function declaration (parameters can only be omitted from right to left)
CREATE FUNCTION concat_lower_or_upper(a text, b text, uppercase boolean DEFAULT false)
RETURNS text AS $$
SELECT CASE WHEN $3 THEN UPPER($1 || ' ' || $2) ELSE LOWER($1 || ' ' || $2) END;
$$ LANGUAGE SQL IMMUTABLE STRICT;
Positional Notation
SELECT concat_lower_or_upper('Hello', 'World', true);
HELLO WORLD
SELECT concat_lower_or_upper('Hello', 'World');
hello world
Named notation
SELECT concat_lower_or_upper(a := 'Hello', b := 'World', uppercase := true);
HELLO WORLD
SELECT concat_lower_or_upper(a := 'Hello', uppercase := true, b := 'World');
HELLO WORLD
Mixed Notation
SELECT concat_lower_or_upper('Hello', 'World', uppercase := true);
HELLO WORLD
5. Data DefinitionConstraints - following example demonstrates different methods of constraints usage.
CREATE TABLE products ( CHECK (price > 0), price numeric, price1 numeric, CONSTRAINT positive_price CHECK (price1 > 0), CHECK (price > price1), CHECK (discounted_price > 0 AND price > discounted_price), product_no integer UNIQUE, product_no integer CONSTRAINT must_be_different UNIQUE, product_no integer REFERENCES products (product_no), -- foreign key FOREIGN KEY (b, c) REFERENCES other_table (c1, c2), product_no integer REFERENCES products ON DELETE RESTRICT, order_id integer REFERENCES orders ON DELETE CASCADE);
RESTRICT prevents deletion of a referenced row. NO ACTION means that if any referencing rows still exist when the constraint is checked. CASCADE specifies that when a referenced row is deleted, row(s) referencing it should be automatically deleted as well.
Modifying TableALTER TABLE products ADD COLUMN description text; --adding a column
ALTER TABLE products DROP COLUMN description CASCADE; -- drop column
-- add constrain
ALTER TABLE products ADD CHECK (name <> '');
ALTER TABLE products ADD CONSTRAINT some_name UNIQUE (product_no);
ALTER TABLE products ADD FOREIGN KEY (product_id) REFERENCES product_groups;
ALTER TABLE products ALTER COLUMN price SET DEFAULT 7.77; -- add default
ALTER TABLE products RENAME COLUMN prod_no TO product_no; -- rename column
6.Partitioning
Partitioning refers to splitting what is logically one large table into smaller physical pieces. Benefits are:
- Query performance can be improved, when heavily accessed rows of table are in a single partition. The partitioning substitutes for leading columns of indexes, reducing index size and making it more likely that the heavily-used parts of the indexes fit in memory.
- When queries or updates access a large percentage of a single partition, performance can be improved by taking advantage of sequential scan of that partition instead of using an index..
- Bulk loads and deletes can be accomplished by adding or removing partitions, if that requirement is planned into the partitioning design.
- Seldom-used data can be migrated to cheaper and slower storage media.
Range Partitioning
The table is partitioned into "ranges" defined by a key column or set of columns, with no overlap
List Partitioning
The table is partitioned by explicitly listing which key values appear in each partition.
Combining Queries
The results of two queries can be combined using the set operations union, intersection, and difference.
- query1 UNION [ALL] query2 - eliminates duplicate rows from its result, in the same way as DISTINCT, unless UNION ALL is used
- query1 INTERSECT [ALL] query2 - returns all rows that are both in the result. Duplicate rows are eliminated unless INTERSECT ALL is used.
- query1 EXCEPT [ALL] query2 - returns all rows that are in the result of query1 but not in the result of query2. (This is sometimes called the difference between two queries.)
7.With Queries (Common Table Expressions)
WITH provides a way to write auxiliary statements for use in a larger query. These statements, which are often referred to as Common Table Expressions or CTEs, can be thought of as defining temporary tables that exist just for one query.
Data-Modifying Statements in WITH
You can use data-modifying statements (INSERT, UPDATE, or DELETE) in WITH. This allows you to perform several different operations in the same query. An example is:
WITH moved_rows AS (
DELETE FROM products
WHERE
"date" >= '2010-10-01' AND
"date" < '2010-11-01'
RETURNING *
)
INSERT INTO products_log
SELECT * FROM moved_rows;
This query effectively moves rows from products to products_log. The DELETE in WITH deletes the specified rows from products, returning their contents by means of itsRETURNING clause; and then the primary query reads that output and inserts it into products_log.
8. Data typeChapter 9. Functions and Operators
9.3. Mathematical Functions and Operatorshttp://www.postgresql.org/docs/9.1/interactive/functions-math.html
9.4. String Functions and Operators
http://www.postgresql.org/docs/9.1/interactive/functions-string.html
9.5. Binary String Functions and Operators
http://www.postgresql.org/docs/9.1/interactive/functions-binarystring.html
9.7. Pattern Matching
http://www.postgresql.org/docs/9.1/interactive/functions-matching.html
9.8. Data Type Formatting Functions
http://www.postgresql.org/docs/9.1/interactive/functions-formatting.html
9.9. Date/Time Functions and Operators
http://www.postgresql.org/docs/9.1/interactive/functions-datetime.html
9.10. Enum Support Functions
http://www.postgresql.org/docs/9.1/interactive/functions-enum.html
http://www.postgresql.org/docs/9.1/interactive/functions-geometry.html
9.12. Network Address Functions and Operators
http://www.postgresql.org/docs/9.1/interactive/functions-net.html
9.13. Text Search Functions and Operators
http://www.postgresql.org/docs/9.1/interactive/functions-textsearch.html
9.14. XML Functions
http://www.postgresql.org/docs/9.1/interactive/functions-xml.html
9.15. Sequence Manipulation Functions
http://www.postgresql.org/docs/9.1/interactive/functions-sequence.html
9.16. Conditional Expressions
Case
SELECT ... WHERE CASE WHEN x <> 0 THEN y/x > 1.5 ELSE false END;
NULLIF
9.17. Array Functions and Operators
The
NULLIF function returns a null value if value1 equals value2; otherwise it returns value1. This can be used to perform the inverse operation of the COALESCE example given above:SELECT NULLIF(value, '(none)')
http://www.postgresql.org/docs/9.1/interactive/functions-array.html
9.18. Aggregate Functions
http://www.postgresql.org/docs/9.1/interactive/functions-aggregate.html
9.19. Window Functions
http://www.postgresql.org/docs/9.1/interactive/functions-window.html
9.22. Set Returning Functions
This section describes functions that possibly return more than one row. http://www.postgresql.org/docs/9.1/interactive/functions-info.html
SELECT * FROM generate_series(2,4);
generate_series
-----------------
2
3
4
SELECT * FROM generate_series(5,1,-2);
generate_series
-----------------
5
3
1
-- this example relies on the date-plus-integer operator
SELECT current_date + s.a AS dates FROM generate_series(0,14,7) AS s(a);
dates
------------
2004-02-05
2004-02-12
2004-02-19
SELECT * FROM generate_series('2008-03-01 00:00'::timestamp, '2008-03-04 12:00', '10 hours');
generate_series
---------------------
2008-03-01 00:00:00
2008-03-01 10:00:00
2008-03-01 20:00:00
2008-03-02 06:00:00
2008-03-02 16:00:00
9.23 System Information Functions
Session Information Functions
Session Information Functions
| Name | Return Type | Description |
|---|---|---|
current_catalog | name | name of current database (called "catalog" in the SQL standard) |
current_database() | name | name of current database |
current_query() | text | text of the currently executing query, as submitted by the client (might contain more than one statement) |
current_schema[()] | name | name of current schema |
current_schemas(boolean) | name[] | names of schemas in search path, optionally including implicit schemas |
current_user | name | user name of current execution context |
inet_client_addr() | inet | address of the remote connection |
inet_client_port() | int | port of the remote connection |
inet_server_addr() | inet | address of the local connection |
inet_server_port() | int | port of the local connection |
pg_backend_pid() | int | Process ID of the server process attached to the current session |
pg_conf_load_time() | timestamp with time zone | configuration load time |
pg_is_other_temp_schema(oid) | boolean | is schema another session's temporary schema? |
pg_listening_channels() | setof text | channel names that the session is currently listening on |
pg_my_temp_schema() | oid | OID of session's temporary schema, or 0 if none |
pg_postmaster_start_time() | timestamp with time zone | server start time |
session_user | name | session user name |
user | name | equivalent to current_user |
version() | text | PostgreSQL version information |
http://www.postgresql.org/docs/9.1/interactive/functions-admin.html
Labels:
Postgres
Subscribe to:
Posts (Atom)