Nov
14
2008
0

SQL Joins and Applies – Understanding APPLY

This is part of a four part post series on SQL Joins and Applies.
If you want to follow the examples given, you can get the T-SQL to create the tables, function and test data from my earlier post SQL Joins and Applies – Test Data.

The other posts in this series are:
SQL Joins and Applies – Understanding INNER JOIN
SQL Joins and Applies – Understanding CROSS JOIN
SQL Joins and Applies – Understanding OUTER JOIN

For some reason APPLY is not well known or understood. It is required when you have a Table-Valued Function (UDF, XML.nodes(), etc) and you need to process a table of values against the function and get a “Joined” result.

From Books Online (For an outer Join):

The APPLY operator allows you to invoke a table-valued function for each row returned by an outer table expression of a query. The table-valued function acts as the right input and the outer table expression acts as the left input. The right input is evaluated for each row from the left input and the rows produced are combined for the final output. The list of columns produced by the APPLY operator is the set of columns in the left input followed by the list of columns returned by the right input.

To simplify matters, let’s first look at timed when an APPLY is not required when using a Table-Valued Function in a query:

SELECT		tsi.sysID [sysID_table],
		tudf.calculatedValue [result_function]
FROM		dbo.tblSourceIntegers tsi
LEFT OUTER JOIN	dbo.udfGet5Rows(2) tudf
ON		tsi.sysID = tudf.calculatedValue

In the above query, the function is called with a constant which means the function can be evaluated to a table BEFORE the join is performed. Any values which do not rely on the rest of the query should work in place of the value of 2 – You can even use a random integer here.

Here is one that uses a value from the table:

SELECT	tsi.sysID [sysID_table]
FROM	dbo.tblSourceIntegers tsi
WHERE	8 IN (SELECT calculatedValue FROM dbo.udfGet5Rows(tsi.sysID))

Although the function is using a value from the main query, this is in the WHERE which happens after the table is loaded into memory and as such.

As soon as the function is being executed for each value in the query as part of the query result, an Apply needs to be used.

There are two types of applies, CROSS APPLY and OUTER APPLY.

  • CROSS APPLY will not “Join” if the Table-Valued Result is NULL
  • OUTER APPLY will “Join” if the Table-Valued Result is NULL

Note: If the table-Valued function returns a column with NULL in it, CROSS APPLY will still perform the “join”. The only time CROSS APPLY will not is when the Table-Valued Result is actually a NULL table.

Here is a sample CROSS APPLY:

SELECT		tsi.sysID [sysID_table],
		applied.calculatedValue
FROM		dbo.tblSourceIntegers tsi
CROSS APPLY	dbo.udfGet5Rows(tsi.sysID) applied

The result of the above Cross Apply is:

sysID_table calculatedValue
----------- ---------------
1           1
1           2
1           NULL
1           4
1           5
2           2
2           3
2           NULL
2           5
2           6
4           4
4           5
4           NULL
4           7
4           8
5           5
5           6
5           NULL
5           8
5           9

(20 row(s) affected)

Note that the NULLs on the Right hand column were one row out of 5 rows in the table returned from the Table-Valued Function.

Now let’s try the OUTER APPLY:

SELECT		tsi.sysID [sysID_table],
		applied.calculatedValue
FROM		dbo.tblSourceIntegers tsi
OUTER APPLY	dbo.udfGet5Rows(tsi.sysID) applied

The result of the above Outer Apply is:

sysID_table calculatedValue
----------- ---------------
1           1
1           2
1           NULL
1           4
1           5
2           2
2           3
2           NULL
2           5
2           6
NULL        NULL
4           4
4           5
4           NULL
4           7
4           8
5           5
5           6
5           NULL
5           8
5           9

(21 row(s) affected)

Note that the OUTER APPLY only returned one more row from when the function was called with a NULL value as the input parameter which the function returned a NULL result.

To review, an APPLY is useful for when you need to apply the values from a table to a Table-Valued function. APPLY is very resource intensive as it is recursive (The function will be run once for every matching value from the first table).

Written by John Burns in: SQL Tips, Tips |
Nov
13
2008
0

SQL Joins and Applies – Understanding OUTER JOIN

This is part of a four part post series on SQL Joins and Applies.
If you want to follow the examples given, you can get the T-SQL to create the tables, function and test data from my earlier post SQL Joins and Applies – Test Data.

The other posts in this series are:
SQL Joins and Applies – Understanding INNER JOIN
SQL Joins and Applies – Understanding CROSS JOIN
SQL Joins and Applies – Understanding APPLY

Outer Joins are joins that include rows even if they do not have related rows in the joined table. There are three types of outer join, each one specifies if you will always see the rows from the LEFT, RIGHT or both (FULL) tables.

From Books Online (For an outer Join):

A join that includes rows even if they do not have related rows in the joined table.

LEFT OUTER JOIN
From Books Online:

All rows from the first-named table (the “left” table, which appears leftmost in the JOIN clause) are included. Unmatched rows in the right table do not appear.

SELECT		tsi.sourceValue [sourceValue_left],
		tsi.sysID [sysID_left],
		tss.sysID [sysID_Right],
		tss.sourceValue [sourceValue_right]
FROM		dbo.tblSourceIntegers tsi
LEFT OUTER JOIN	dbo.tblSourceStrings tss
	ON	tsi.sysID = tss.sysID;

Results:

sourceValue_left sysID_left  sysID_Right sourceValue_right
---------------- ----------- ----------- -----------------
100              1           NULL        NULL
200              2           2           Apple
NULL             NULL        NULL        NULL
400              4           NULL        NULL
500              5           5           Dog

(5 row(s) affected)

As above, all of the values in the left table were returned.

RIGHT OUTER JOIN
From Books Online:

All rows in the second-named table (the “right” table, which appears rightmost in the JOIN clause) are included. Unmatched rows in the left table are not included.

SELECT			tsi.sourceValue [sourceValue_left],
			tsi.sysID [sysID_left],
			tss.sysID [sysID_Right],
			tss.sourceValue [sourceValue_right]
FROM			dbo.tblSourceIntegers tsi
RIGHT OUTER JOIN	dbo.tblSourceStrings tss
	ON		tsi.sysID = tss.sysID;

Results:

sourceValue_left sysID_left  sysID_Right sourceValue_right
---------------- ----------- ----------- -----------------
200              2           2           Apple
NULL             NULL        3           Banana
NULL             NULL        NULL        NULL
500              5           5           Dog
NULL             NULL        6           Egg

(5 row(s) affected)

As above, all of the values in the right table were returned.

FULL OUTER JOIN
From Books Online:

All rows in all joined tables are included, whether they are matched or not.

SELECT		tsi.sourceValue [sourceValue_left],
		tsi.sysID [sysID_left],
		tss.sysID [sysID_Right],
		tss.sourceValue [sourceValue_right]
FROM		dbo.tblSourceIntegers tsi
FULL OUTER JOIN	dbo.tblSourceStrings tss
	ON	tsi.sysID = tss.sysID;

Results:

sourceValue_left sysID_left  sysID_Right sourceValue_right
---------------- ----------- ----------- -----------------
100              1           NULL        NULL
200              2           2           Apple
NULL             NULL        NULL        NULL
400              4           NULL        NULL
500              5           5           Dog
NULL             NULL        3           Banana
NULL             NULL        NULL        NULL
NULL             NULL        6           Egg

(8 row(s) affected)

Looking at the above result, all of the values in the left table were returned first, followed by the unmatched values from the right table.

Written by John Burns in: SQL Tips, Tips |
Nov
12
2008
0

SQL Joins and Applies – Understanding CROSS JOIN

This is part of a four part post series on SQL Joins and Applies.
If you want to follow the examples given, you can get the T-SQL to create the tables, function and test data from my earlier post SQL Joins and Applies – Test Data.

The other posts in this series are:
SQL Joins and Applies – Understanding INNER JOIN
SQL Joins and Applies – Understanding OUTER JOIN
SQL Joins and Applies – Understanding APPLY

Cross join is probably the least useful join in a relational database.

From Books Online:

A join whose result set includes one row for each possible pairing of rows from the two tables.

Warning:
Cross Join can use a lot of memory when applied to large tables. Every row in the left table is repeated for every row of the right table (Unless you use WHERE to filter the results)

Here is a sample cross join:

SELECT		tsi.sourceValue [sourceValue_left],
		tsi.sysID [sysID_left],
		tss.sysID [sysID_Right],
		tss.sourceValue [sourceValue_right]
FROM		dbo.tblSourceIntegers tsi
CROSS JOIN	dbo.tblSourceStrings tss;

And here is the large result set:

sourceValue_left sysID_left  sysID_Right sourceValue_right
---------------- ----------- ----------- -----------------
100              1           2           Apple
200              2           2           Apple
NULL             NULL        2           Apple
400              4           2           Apple
500              5           2           Apple
100              1           3           Banana
200              2           3           Banana
NULL             NULL        3           Banana
400              4           3           Banana
500              5           3           Banana
100              1           NULL        NULL
200              2           NULL        NULL
NULL             NULL        NULL        NULL
400              4           NULL        NULL
500              5           NULL        NULL
100              1           5           Dog
200              2           5           Dog
NULL             NULL        5           Dog
400              4           5           Dog
500              5           5           Dog
100              1           6           Egg
200              2           6           Egg
NULL             NULL        6           Egg
400              4           6           Egg
500              5           6           Egg

(25 row(s) affected)

Note:
The following two queries are identical both in their result set AND in execution plan.

SELECT		tsi.sourceValue [sourceValue_left],
		tsi.sysID [sysID_left],
		tss.sysID [sysID_Right],
		tss.sourceValue [sourceValue_right]
FROM		dbo.tblSourceIntegers tsi
INNER JOIN	dbo.tblSourceStrings tss
	ON	tsi.sysID = tss.sysID;
 
SELECT		tsi.sourceValue [sourceValue_left],
		tsi.sysID [sysID_left],
		tss.sysID [sysID_Right],
		tss.sourceValue [sourceValue_right]
FROM		dbo.tblSourceIntegers tsi
CROSS JOIN	dbo.tblSourceStrings tss
WHERE		tsi.sysID = tss.sysID;
Written by John Burns in: SQL Tips, Tips |
Nov
11
2008
0

SQL Joins and Applies – Understanding INNER JOIN

Understanding INNER JOIN.

The other posts in this series are:
SQL Joins and Applies – Understanding CROSS JOIN
SQL Joins and Applies – Understanding OUTER JOIN
SQL Joins and Applies – Understanding APPLY

This is part of a four part post series on SQL Joins and Applies.
If you want to follow the examples given, you can get the T-SQL to create the tables, function and test data from my earlier post SQL Joins and Applies – Test Data.

Inner join is the most useful join to know in a relational database.

From Books Online:

A join that displays only the rows that have a match in both joined tables.

Lets jump straight into some SQL.

SELECT		tsi.sourceValue [sourceValue_left],
		tsi.sysID [sysID_left],
		tss.sysID [sysID_Right],
		tss.sourceValue [sourceValue_right]
FROM		dbo.tblSourceIntegers tsi
INNER JOIN	dbo.tblSourceStrings tss
	ON	tsi.sysID = tss.sysID;

The result of this is:

sourceValue_left sysID_left  sysID_Right sourceValue_right
---------------- ----------- ----------- -----------------
200              2           2           Apple
500              5           5           Dog

(2 row(s) affected)

Note that the use of a comparison
In the above example, the comparison operator is comparing the sysID from both tables.
Results are only returned if the sysID’s match. Remember that nothing (not even another NULL) will evaluate to true when compared with another NULL.

A comparison operator is mandatory in an inner join.

Note: The use of “ON 1=1″ is identical to a CROSS JOIN

Written by John Burns in: SQL Tips, Tips |
Nov
10
2008
0

Creating ISOs in Mac OS X

Here are step by step instructions on how to create an ISO from a CD or DVD in Mac OS X.

First, insert the CD/DVD and open terminal.

Type in:

drutil status

Your results may vary, here is what I see.

Last login: Mon Nov 10 09:45:51 on console
JBMac:~ John$ drutil status
 Vendor   Product           Rev
 MATSHITA DVD-R   UJ-857E   ZA0E

         Type: CD-ROM               Name: /dev/disk1
     Sessions: 1                  Tracks: 1
 Overwritable:   00:00:00     blocks:        0 /   0.00MB /   0.00MiB
   Space Free:   00:00:00     blocks:        0 /   0.00MB /   0.00MiB
   Space Used:   05:10:14     blocks:    23264 /  47.64MB /  45.44MiB
  Writability:

JBMac:~ John$

The part we are interested in is Name: /dev/disk1
If your drive is not /dev/disk1, you will need to use your drive’s name.

We now need to unmount the drive, this is done by typing:

diskutil unmountDisk /dev/disk1

You should see something like:

JBMac:~ John$ diskutil unmountDisk /dev/disk1
Unmount of all volumes on disk1 was successful
JBMac:~ John$ 

Now all we need to do is create the iso file. To do this we use the utility dd.

dd if=/dev/disk1 of=outputfile.iso bs=2048

Depending on the size of the disk, this may take some time. Once complete, you should see something along the lines of.

JBMac:~ John$ dd if=/dev/disk1 of=outputfile.iso bs=2048
26717+1 records in
26717+1 records out
54716928 bytes transferred in 60.473826 secs (904803 bytes/sec)
JBMac:~ John$

And your ISO file should be created. Test it with finder or using hdid in the terminal

Written by John Burns in: Mac OS X Tips |
Nov
10
2008
0

SQL Joins and Applies – Test Data

I’m writing a four part series on Joins and Applies in SQL.

Below is the T-SQL to create the two tables and a function used to explain SQL Joins and Applies.

--CREATE A TABLE VALUED FUNCTION
--FUNCTION IS ONLY USED FOR CROSS AND OUTER APPLY
--CREATE A TABLE VALUED FUNCTION
--FUNCTION IS ONLY USED FOR CROSS AND OUTER APPLY
CREATE FUNCTION [dbo].[udfGet5Rows](@inputInteger INT)
RETURNS @t TABLE ([calculatedValue] INT) AS
BEGIN
	IF @inputInteger IS NOT NULL
	BEGIN
		INSERT INTO @t([calculatedValue])
		SELECT @inputInteger		UNION ALL
		SELECT @inputInteger + 1	UNION ALL
		SELECT NULL			UNION ALL
		SELECT @inputInteger + 3	UNION ALL
		SELECT @inputInteger + 4
 	END
 
	RETURN
END;
GO
--CREATE TWO TABLES
CREATE TABLE [dbo].[tblSourceIntegers]
(
sysID INT NULL,
sourceValue INT NULL
);
GO
 
CREATE TABLE [dbo].[tblSourceStrings]
(
sysID INT NULL,
sourceValue VARCHAR(16) NULL
);
GO
 
--INSERT VALUES INTO THE TABLES
INSERT INTO tblSourceIntegers(sysID,sourceValue)
	SELECT 1,	100		UNION ALL
	SELECT 2,	200		UNION ALL
	SELECT NULL,	NULL		UNION ALL
	SELECT 4,	400		UNION ALL
	SELECT 5,	500;
GO
 
INSERT INTO tblSourceStrings(sysID,sourceValue)
	SELECT 2,	'Apple'		UNION ALL
	SELECT 3,	'Banana'	UNION ALL
	SELECT NULL,	NULL		UNION ALL
	SELECT 5,	'Dog'		UNION ALL
	SELECT 6,	'Egg';
GO

Test your results with:

SELECT * FROM dbo.udfGet5Rows(5);
SELECT * FROM dbo.tblSourceIntegers;
SELECT * FROM tblSourceStrings;

Your results should be:

calculatedValue
---------------
5
6
NULL
8
9

(5 row(s) affected)

sysID       sourceValue
----------- -----------
1           100
2           200
NULL        NULL
4           400
5           500

(5 row(s) affected)

sysID       sourceValue
----------- ----------------
2           Apple
3           Banana
NULL        NULL
5           Dog
6           Egg

(5 row(s) affected)
Written by John Burns in: SQL Tips, Tips |
Nov
06
2008
0

High Performance Bulk Inserts into SQL – Considerations

Did you know that simply setting the Database Recovery Model to BULK_LOGGED does not guarantee minimal logging when inserting?

At the very least, for performance:

  • Recovery Model should be set to BULK_LOGGED.
  • Destination table should not be replicated.
  • Table must be empty if indexes are on it.
  • TABLOCK should be used (Where applicable).
  • Import should be done during times of low DB load or during specified downtime.

The following table describes logging that will happen during an insert when the Recovery Model is BULK_LOGGED.

  Non Clustered Index Clustered Index
  Yes No Yes No
Table Has Data Index Minimal Full Minimal
Table Empty Minimal Minimal Minimal Minimal

Remember also that if you are inserting in batches, only the first batch is into an empty table, the other inserts are into a table with data. The best solution is to always insert into an empty table with no indexes with the BULK_LOGGED Recovery model set.

Checking the recovery model is as easy as

SELECT DATABASEPROPERTYEX('[databasename]','Recovery');

If you are using SQL Server Enterprise Edition, remember that you can rebuild indexes ONLINE. This means that indexes can be rebuilt without locking the table. This means users can still perform searches (naturally without the indexes) while the indexes are rebuilding.

Written by John Burns in: SQL Tips, Tips |
Nov
04
2008
0

Covering Indexes in SQL 2005 – Results

After a few queries regarding my recent post on Covering Indexes, I decided to do a couple of tests to show you the results.

Below are the actual execution plan results using the table structure and SQL query from the earlier post. The table had three rows of data.

Results WITHOUT a covering index

Results WITHOUT a covering index

Results WITH a covering index

Results WITH a covering index

As you can (hopefully) see, the covering index resulted in a simpler execution plan.

If we compare the two subtree costs of 0.0032831 (Covering index) and 0.0065704 (No covering index) we get a performance gain of 2.001 or almost bang on 200% improvement.

I hope this helps

Written by John Burns in: SQL Tips, Tips |
Nov
04
2008
0

Screenshots in Mac OS X from the terminal

Following on from yesterdays post about screen shots, if you want to script a screen grab, the following command will work.

screencapture ~/Desktop/filename.jpg

Remember that you can get the list of options by using man:
man screencapture

Written by John Burns in: Mac OS X Tips |
Nov
03
2008
0

Screenshots in Mac OS X

The following key-presses will take a screen grab and save it to the desktop.  Add CTRL to the mix to save it to the clipboard for later pasting.

Full Screen
Command-Shift-3.

Selected Area
Command-Shift-4 Then click and drag to select the area to capture.

Selected window
Command-Shift-4 Then [space] Then click the window to capture.

Written by John Burns in: Mac OS X Tips |