5/10/2018. Connor McDonald

Transcription

1 1 Connor McDonald 1

2 3 4 2

3 connor-mcdonald.com 5 asktom.oracle.com 6 3

4 why talk about SQL? # 1 7 after all

5 9 NoSQL non relational 10 5

6 why talk about SQL? # 2 11 SQL invented "cool" 12 6

7 MICROSERVICES 13 "fine-grained to perform a single function" "Each service is... minimal, and complete" select COUNT(*) from PEOPLE where GENDER = 'MALE'

8 API 15 "By abstracting the underlying implementation" "describes the expected behaviour... but can have multiple implementations" select NAME, STREET_NO, ZIP_CODE from PEOPLE p, ADDRESS a where p.age > 50 and p.address_id = a.address_id

9 why talk about SQL? # 3 17 developers 18 9

10 coding for functionality

11 21... not for data processing 22 11

12 23 a little bit of SQL

13 ... can make your apps awesome 25 key point 26 13

14 this session is about smart not ego 28 14

15 SQL can do anything SQL> with x( s, ind ) as 2 ( select sud, instr( sud, '.' ) 3 from ( select replace(replace( 4 replace(replace(:board,'-'),' '),' '),chr(10)) sud 5 from dual ) 6 union all 7 select substr(s,1,ind-1) z substr(s,ind+1) 8, instr(s,'.',ind+1) 9 from x 10, ( select to_char( rownum ) z 11 from dual connect by rownum <= 9 ) z 12 where ind > 0 13 and not exists ( 14 select null 15 from ( select rownum lp from dual 16 connect by rownum <= 9 ) 17 where z = substr(s,trunc((ind-1)/9)*9+lp,1) 30 15

16 31 18 or z = substr(s,mod(ind-1,9)-8+lp*9,1) 19 or z = substr(s,mod(trunc((ind-1)/3),3)*3 20 +trunc((ind-1)/27)*27+lp 21 +trunc((lp-1)/3)*6,1) 22 ) 23 ), 24 result as ( 25 select s 26 from x 27 where ind = 0 ) 28 select 29 regexp_replace(substr(s,(idx-1)*9+1,9), 30 '(...)(...)(...)', 31 '\1 \2 \3') 32 case when mod(idx,3)=0 then chr(10) rpad('-',11,'-') end soln 33 from result, 34 ( select level idx 35 from dual 36 connect by level <= 9 ) Ack: Anton Scheffer, SQL> variable board varchar2(1000) SQL> begin :board := 2 ' '; 14 end;

17 SOLUTION sud.sql % % of developers that need to solve Sudoku 34 17

18 100% % of developers that need to get real work done 35 SQL for real work 36 18

19 1 37 important relationship 38 19

20 DBA 39 20

21 versus Public 41 the DBA can help oc00.sql 42 21

22 2 43 generating rows 44 22

23 insert into my_transactions1 select... from tab200k; "What if I don't have a TAB200K?" 45 the problem

24 only just enough databases store data 47 SQL> select * from meetings; SCHEDULED OWNER MAY-17 Connor 13-MAY-17 Gerald 17-MAY-17 Maria 48 24

25 free free free free free Connor free free free free free free Gerald free free free Maria free free free free free free free free free free free free free free 49 what we need 50 25

26 SQL> select * from meetings; SCHEDULED OWNER MAY MAY MAY MAY MAY MAY-17 Connor 07-MAY MAY MAY MAY-17 Maria 18-MAY MAY MAY MAY but how many? 31? 365? 52 26

27 any number of rows 53 easy 54 27

28 select * from generate_series(1, 10) Postgres with gen(seq) as ( select 1 seq union all select seq+1 from gen where seq < 10 ) select * from gen; SQL Server select seq from my_n_row_table MySQL 55 SQL> select level seq 2 from DUAL 3 connect by level <= 10; SEQ rows selected

29 SQL> select level seq 2 from DUAL 3 connect by level <= 10; Id Operation Name Rows SELECT STATEMENT 1 * 1 CONNECT BY WITHOUT FILTERING 2 FAST DUAL Statistics recursive calls 0 db block gets 0 consistent gets 0 physical reads 57 "free" 58 29

30 if I have numbers I have dates 60 30

31 SQL> select date ' '+level dte 2 from dual 3 connect by level <= 31; DTE MAY MAY MAY MAY MAY MAY MAY rows selected. 61 use an outer join 62 31

32 SQL> select 2 dte scheduled, SCHEDULED OWNER 3 m.owner from 01-MAY-17 5 ( select date ' '+level 02-MAY-17 dte 6 from dual 03-MAY-17 7 connect by level 04-MAY-17 <= 31 ) d 8 left outer join 05-MAY-17 9 meetings m 06-MAY-17 Connor 10 on d.dte = m.scheduled 07-MAY order by 1; MAY MAY MAY-17 Maria 18-MAY MAY MAY MAY

33 subquery factoring 65 common table expressions 66 33

34 WITH clause 67 SQL> WITH last_hire AS 2 ( select deptno, max(hiredate) 3 from emp 4 group by deptno 5 ) 6 select * from last_hire; DEPTNO MAX(HIRED DEC JAN JAN

35 "Isn't this just more code?" 69 why is it cool? 70 35

36 good way to build complex SQL 71 relational is a rigorous model

37 relational is the dominant model but relational... is complex! 74 37

38 not our fault 75 Codd & Date 76 38

39 ? "data is represented as mathematical n-ary relations, an n-ary relation being a subset of the Cartesian product of n domains." 77 procedural world 78 39

40 "First, get the total salary paid by each department, then get the average of these totals, then list those departments above that average" SQL? 79 "First, get the total salary paid by department... SQL> WITH dept_salaries AS ( 2 SELECT dname, SUM(sal) dept_sal 3 FROM emp e, dept d 4 WHERE e.deptno = d.deptno 5 GROUP BY dname), 80 40

41 "...then get the average of these totals... 6 avg_sal AS ( SELECT AVG(dept_sal) avsal 7 FROM dept_salaries) 81 "...then list those departments above average." 8 SELECT * FROM dept_salaries d, avg_sal a 9 WHERE d.dept_sal > a.avsal 10 ORDER BY d.dname; 82 41

42 SQL> WITH dept_salaries AS ( 2 SELECT dname, SUM(sal) dept_sal 3 FROM emp e, dept d 4 WHERE e.deptno = d.deptno 5 GROUP BY dname), 6 avg_sal AS ( SELECT AVG(dept_sal) avsal 7 FROM dept_salaries) 8 SELECT * FROM dept_salaries d, avg_sal a 9 WHERE d.dept_sal > a.avsal 10 ORDER BY d.dname; 83 we took programmer's approach

43 got a relational solution 85 good for code reuse 86 43

44 recall : 31 days of meetings 87 SQL> with raw_data as ( 2 select 3 dte scheduled, [{"01-MAY-17":null}, 4 m.owner {"02-MAY-17":null}, 5 from {"03-MAY-17":null}, 6 ( select date ' '+level {"04-MAY-17":null}, dte 7 from dual {"05-MAY-17":null}, 8 connect by level {"06-MAY-17":"Connor"}, <= 31 ) d 9 left outer join {"07-MAY-17":null}, 10 meetings m {"08-MAY-17":null}, 11 on d.dte = m.scheduled {"09-MAY-17":null}, 12 ) {"10-MAY-17":null}, 13 select json_arrayagg( json_object(key... to_char(scheduled) value owner ) 16 order by scheduled {"30-MAY-17":null}, ) as meetings 17 from raw_data ; {"31-MAY-17":null} ] 88 44

45 4 89 partitioned outer join 90 45

46 SQL> select * 2 from room_times; HR SQL> select * 2 from bookings; HR ROOM WHO Room2 PETE 9 Room1 JOHN 11 Room1 MIKE 14 Room2 JILL 15 Room2 JANE 16 Room1 SAM 91 if I need bookings by hour conventional outer join 92 46

47 SQL> SELECT hrs.hr, t1.room, t1.who 2 from room_times hrs 3 left outer join bookings t1 4 on hrs.hr = t1.hr HR ROOM WHO Room2 PETE 9 Room1 JOHN Room1 MIKE Room2 JILL 15 Room2 JANE 16 Room1 SAM bookings by hour per room 94 47

48 HR ROOM WHO Room1 JOHN Room1 MIKE Room1 SAM HR ROOM WHO Room2 PETE Room2 JILL 15 Room2 JANE SQL> select * 2 from room_times; HR x "Room 1" x "Room 2"... x "Room n" 96 48

49 partitioned outer join 97 SQL> SELECT hrs.hr, t1.room, t1.who 2 FROM bookings t1 3 PARTITION BY (t1.room) 4 RIGHT OUTER JOIN room_times ON (hrs.hr = t1.hr); HR ROOM WHO Room1 9 Room1 JOHN 10 Room1 11 Room1 MIKE 12 Room1 13 Room1 14 Room1 15 Room1 16 Room1 SAM 8 Room2 PETE 9 Room2 10 Room2 11 Room2 12 Room2 13 Room2 14 Room2 JILL 15 Room2 JANE 16 Room

50 5 99 pagination

51 "employees by hiredate, more recent first" 101 SQL> select empno, ename, hiredate 2 from emp 3 limit 5 4 order by hiredate desc; EMPNO ENAME HIREDATE MARTIN 28/09/ :00: JONES 02/04/ :00: WARD 22/02/ :00: ALLEN 20/02/ :00: SMITH 17/12/ :00:

52 SQL> select empno, ename, hiredate 2 from emp 3 where rownum <= 5 4 order by hiredate desc; EMPNO ENAME HIREDATE MARTIN 28/09/ :00: JONES 02/04/ :00: WARD 22/02/ :00: ALLEN 20/02/ :00: SMITH 17/12/ :00: inline view

53 SQL> select * 2 from ( 3 select empno, ename, hiredate 4 from emp 5 order by hiredate desc 6 ) 7 where rownum <= 5; EMPNO ENAME HIREDATE ADAMS 12-JAN SCOTT 09-DEC MILLER 23-JAN JAMES 03-DEC FORD 03-DEC SQL> select * 2 from ( 3 select 4 empno, ename, hiredate, 5 row_number() over ( order by hiredate desc) rn 6 from emp 7 ) 8 where rn <= 5;

54 SQL> select empno, ename, hiredate 2 from emp 3 order by hiredate desc 4 fetch first 5 rows only; EMPNO ENAME HIREDATE ADAMS 12-JAN SCOTT 09-DEC MILLER 23-JAN JAMES 03-DEC FORD 03-DEC "TL;DR... do it in application code"

55 public static void Paging(Connection conn ) throws Exception { PreparedStatement sql_stmt = conn.preparestatement( "select empno, ename, hiredate from emp order by hiredate desc"); } ResultSet rset = sql_stmt.executequery(); int i = 0; while( rset.next() ) {... i = i + 1; if (i > 5) { break; } } rset.close();

56 demo oc01a.sql 111 let the database know

57 SQL> select * 2 from ( 3 select empno, ename, hiredate 4 from emp 5 order by hiredate desc ) Id Operation Name Rows 7 where rownum <= 5; SELECT STATEMENT 5 * 1 COUNT STOPKEY 2 VIEW 14 * 3 SORT ORDER BY STOPKEY 14 4 TABLE ACCESS FULL EMP SQL> select empno, ename, hiredate 2 from emp 3 order by hiredate desc 4 fetch first 5 rows only; Id Operation Name Rows SELECT STATEMENT 14 * 1 VIEW 14 * 2 WINDOW SORT PUSHED RANK 14 3 TABLE ACCESS FULL EMP

58 you get other benefits oc02.sql 115 for the next page

59 new query 117 SQL> select empno, ename, hiredate 2 from emp 3 order by hiredate desc 4 offset 5 rows fetch first 5 rows only; EMPNO ENAME HIREDATE KING 17-NOV MARTIN 28-SEP TURNER 08-SEP CLARK 09-JUN BLAKE 01-MAY

60 be careful with the OFFSET clause 119 SQL> select empno, ename, hiredate 2 from emp 3 order by hiredate desc; HIREDATE EMPNO ENAME FEB BROWN 12-JAN ADAMS 09-DEC SCOTT 23-JAN MILLER 03-DEC FORD 03-DEC JAMES 17-NOV KING 28-SEP MARTIN 08-SEP TURNER 09-JUN CLARK 01-MAY BLAKE 02-APR JONES 22-FEB WARD 20-FEB ALLEN 17-DEC SMITH fetch first 5 rows offset 5 fetch next

61 SQL> select empno, ename, hiredate 2 from emp 3 order by hiredate desc; HIREDATE EMPNO ENAME FEB BROWN 12-JAN ADAMS 09-DEC SCOTT 23-JAN MILLER 03-DEC FORD 03-DEC JAMES 17-NOV KING 28-SEP MARTIN 08-SEP TURNER 09-JUN CLARK 01-MAY BLAKE 02-APR JONES 22-FEB WARD 20-FEB ALLEN 17-DEC SMITH SQL> select empno, ename, hiredate 2 from emp 3 where hiredate < :last_shown 3 order by hiredate desc; 121 "an expensive query per page?!?!

62 perhaps use result caching 123 SQL> with first_200 as 2 ( select f.*, /*+ result_cache rownum r */ rownum r, f.* 3 from 4 ( select * 5 from t 6 order by owner, object_name desc 7 ) f 8 where rownum <= ) 10 select * 11 from first_ where r between <= and 20 oc03.sql

63 6 125 totals / subtotals

64 "Employee salary list, plus department total, plus grand total" 127 SQL> select empno, ename, sal, deptno from emp 2 order by deptno; EMPNO SQL> ENAME select deptno, SAL DEPTNO sum(sal) CLARK 3 from emp KING 4 group by deptno MILLER 5 order by deptno; JAMES DEPTNO SUM(SAL) BLAKE MARTIN SQL> 10 select 8750 sum(sal) 1250 from 30 emp; SUM(SAL)

65 from 3 to 2 rollup 129 SQL> select empno, ename, sal, deptno from emp 2 order by deptno; EMPNO ENAME SAL DEPTNO SQL> select deptno, CLARK 2 sum(sal) KING 3 from emp MILLER 4 group by rollup(deptno) order by deptno; 7900 JAMES BLAKE DEPTNO SUM(SAL) MARTIN

66 still messy / complicated 131 EMPNO SAL DEPTNO DEPTNO SUM(SAL)

67 from 2 to SQL> select deptno, 2 nvl2(rownum,max(empno),null) empno, 3 nvl2(rownum,max(ename),null) ename, 4 sum(sal) 5 from emp 6 group by rollup(deptno,rownum) 7 order by deptno,empno; 134 DEPTNO EMPNO ENAME SUM(SAL) CLARK KING MILLER SMITH JONES JAMES

68 many more options 135 SQL> select deptno,job,sum(sal) from scott.emp 2 group by CUBE(deptno,job) 3 order by deptno,job; 136 DEPTNO JOB SUM(SAL) CLERK MANAGER PRESIDENT ANALYST CLERK MANAGER CLERK MANAGER SALESMAN ANALYST 6000 CLERK 4150 MANAGER 8275 PRESIDENT 5000 SALESMAN

69 totally customisable 137 SQL> select deptno, job, mgr, sum(sal) from emp 2 group by grouping sets ( 3 (deptno), 4 (job,mgr), () ) ; DEPTNO JOB MGR SUM(SAL) CLERK PRESIDENT 5000 CLERK CLERK CLERK SALESMAN MANAGER ANALYST

70 last 139 staying up to date

71 asktom.oracle.com 141 "fill in the blanks"

72 SQL> select dt, val 2 from t 3 order by dt; DT VAL JAN JAN JAN JAN JAN JAN JAN JAN JAN JAN SQL> select dt, val, 2 case when val is not null 3 then to_char(row_number() over (order by dt),'fm0000') '-' val 4 end max_val 5 from t 6 order by dt; DT VAL MAX_VAL JAN JAN JAN JAN JAN JAN JAN JAN JAN JAN

73 SQL> select dt, val, 2 max(max_val) over (order by dt) max_val_str 3 from ( select dt, val, 4 case when val is not null 5 then to_char(row_number() over (order by dt),'fm0000') '-' val 6 end max_val 7 from t ) order by dt; DT VAL MAX_VAL_STR JAN JAN JAN JAN JAN JAN JAN JAN JAN JAN SQL> select dt, val, 2 to_number(substr(max(max_val) over (order by dt),5)) max_val 3 from ( select dt, val, 4 case when val is not null 5 then to_char(row_number() over (order by dt),'fm0000') val 6 end max_val 7 from t ) order by dt 8 / DT VAL MAX_VAL JAN JAN JAN JAN JAN JAN JAN JAN JAN JAN

74 SQL> select dt, val, 2 last_value(val ignore nulls) over (order by dt) val 3 from t 4 order by dt; DT VAL VAL JAN JAN JAN JAN JAN JAN JAN JAN JAN JAN wrap up

75 SQL 149 very cool

76 very powerful 151 defacto language polyglot world

77 less code 153 never too early to start

78 Thankyou!!! connor-mcdonald. com