postgresql insert into select无法使用并行查询的解决
本文信息基于PG13.1。
从PG9.6开始支持并行查询。PG11开始支持CREATE TABLE … AS、SELECT INTO以及CREATE MATERIALIZED VIEW的并行查询。
先说结论:
换用create table as 或者select into或者导入导出。
首先跟踪如下查询语句的执行计划:
select count(*) from test t1,test1 t2 where t1.id = t2.id ;
postgres=# explain analyze select count(*) from test t1,test1 t2 where t1.id = t2.id ; QUERY PLAN -------------------------------------------------------------------------------------------------------------------------------------------------------- Finalize Aggregate (cost=34244.16..34244.17 rows=1 width=8) (actual time=683.246..715.324 rows=1 loops=1) -> Gather (cost=34243.95..34244.16 rows=2 width=8) (actual time=681.474..715.311 rows=3 loops=1) Workers Planned: 2 Workers Launched: 2 -> Partial Aggregate (cost=33243.95..33243.96 rows=1 width=8) (actual time=674.689..675.285 rows=1 loops=3) -> Parallel Hash Join (cost=15428.00..32202.28 rows=416667 width=0) (actual time=447.799..645.689 rows=333333 loops=3) Hash Cond: (t1.id = t2.id) -> Parallel Seq Scan on test t1 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.025..74.010 rows=333333 loops=3) -> Parallel Hash (cost=8591.67..8591.67 rows=416667 width=4) (actual time=260.052..260.053 rows=333333 loops=3) Buckets: 131072 Batches: 16 Memory Usage: 3520kB -> Parallel Seq Scan on test1 t2 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.032..104.804 rows=333333 loops=3) Planning Time: 0.420 ms Execution Time: 715.447 ms (13 rows)
可以看到走了两个Workers。
下边看一下insert into select:
postgres=# explain analyze insert into va select count(*) from test t1,test1 t2 where t1.id = t2.id ; QUERY PLAN -------------------------------------------------------------------------------------------------------------------------------------------------- Insert on va (cost=73228.00..73228.02 rows=1 width=4) (actual time=3744.179..3744.187 rows=0 loops=1) -> Subquery Scan on "*SELECT*" (cost=73228.00..73228.02 rows=1 width=4) (actual time=3743.343..3743.352 rows=1 loops=1) -> Aggregate (cost=73228.00..73228.01 rows=1 width=8) (actual time=3743.247..3743.254 rows=1 loops=1) -> Hash Join (cost=30832.00..70728.00 rows=1000000 width=0) (actual time=1092.295..3511.301 rows=1000000 loops=1) Hash Cond: (t1.id = t2.id) -> Seq Scan on test t1 (cost=0.00..14425.00 rows=1000000 width=4) (actual time=0.030..421.537 rows=1000000 loops=1) -> Hash (cost=14425.00..14425.00 rows=1000000 width=4) (actual time=1090.078..1090.081 rows=1000000 loops=1) Buckets: 131072 Batches: 16 Memory Usage: 3227kB -> Seq Scan on test1 t2 (cost=0.00..14425.00 rows=1000000 width=4) (actual time=0.021..422.768 rows=1000000 loops=1) Planning Time: 0.511 ms Execution Time: 3745.633 ms (11 rows)
可以看到并没有Workers的指示,没有启用并行查询。
即使开启强制并行,也无法走并行查询。
postgres=# set force_parallel_mode =on; SET postgres=# explain analyze insert into va select count(*) from test t1,test1 t2 where t1.id = t2.id ; QUERY PLAN -------------------------------------------------------------------------------------------------------------------------------------------------- Insert on va (cost=73228.00..73228.02 rows=1 width=4) (actual time=3825.042..3825.049 rows=0 loops=1) -> Subquery Scan on "*SELECT*" (cost=73228.00..73228.02 rows=1 width=4) (actual time=3824.976..3824.984 rows=1 loops=1) -> Aggregate (cost=73228.00..73228.01 rows=1 width=8) (actual time=3824.972..3824.978 rows=1 loops=1) -> Hash Join (cost=30832.00..70728.00 rows=1000000 width=0) (actual time=1073.587..3599.402 rows=1000000 loops=1) Hash Cond: (t1.id = t2.id) -> Seq Scan on test t1 (cost=0.00..14425.00 rows=1000000 width=4) (actual time=0.034..414.965 rows=1000000 loops=1) -> Hash (cost=14425.00..14425.00 rows=1000000 width=4) (actual time=1072.441..1072.443 rows=1000000 loops=1) Buckets: 131072 Batches: 16 Memory Usage: 3227kB -> Seq Scan on test1 t2 (cost=0.00..14425.00 rows=1000000 width=4) (actual time=0.022..400.624 rows=1000000 loops=1) Planning Time: 0.577 ms Execution Time: 3825.923 ms (11 rows)
原因在官方文档有写:
The query writes any data or locks any database rows. If a query contains a data-modifying operation either at the top level or within a CTE, no parallel plans for that query will be generated. As an exception, the commands CREATE TABLE … AS, SELECT INTO, and CREATE MATERIALIZED VIEW which create a new table and populate it can use a parallel plan.
解决方案有如下三种:
1.select into
postgres=# explain analyze select count(*) into vaa from test t1,test1 t2 where t1.id = t2.id ; QUERY PLAN -------------------------------------------------------------------------------------------------------------------------------------------------------- Finalize Aggregate (cost=34244.16..34244.17 rows=1 width=8) (actual time=742.736..774.923 rows=1 loops=1) -> Gather (cost=34243.95..34244.16 rows=2 width=8) (actual time=740.223..774.907 rows=3 loops=1) Workers Planned: 2 Workers Launched: 2 -> Partial Aggregate (cost=33243.95..33243.96 rows=1 width=8) (actual time=731.408..731.413 rows=1 loops=3) -> Parallel Hash Join (cost=15428.00..32202.28 rows=416667 width=0) (actual time=489.880..700.830 rows=333333 loops=3) Hash Cond: (t1.id = t2.id) -> Parallel Seq Scan on test t1 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.033..87.479 rows=333333 loops=3) -> Parallel Hash (cost=8591.67..8591.67 rows=416667 width=4) (actual time=266.839..266.840 rows=333333 loops=3) Buckets: 131072 Batches: 16 Memory Usage: 3520kB -> Parallel Seq Scan on test1 t2 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.058..106.874 rows=333333 loops=3) Planning Time: 0.319 ms Execution Time: 783.300 ms (13 rows)
2.create table as
postgres=# explain analyze create table vb as select count(*) from test t1,test1 t2 where t1.id = t2.id ; QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------------------- Finalize Aggregate (cost=34244.16..34244.17 rows=1 width=8) (actual time=540.120..563.733 rows=1 loops=1) -> Gather (cost=34243.95..34244.16 rows=2 width=8) (actual time=537.982..563.720 rows=3 loops=1) Workers Planned: 2 Workers Launched: 2 -> Partial Aggregate (cost=33243.95..33243.96 rows=1 width=8) (actual time=526.602..527.136 rows=1 loops=3) -> Parallel Hash Join (cost=15428.00..32202.28 rows=416667 width=0) (actual time=334.532..502.793 rows=333333 loops=3) Hash Cond: (t1.id = t2.id) -> Parallel Seq Scan on test t1 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.018..57.819 rows=333333 loops=3) -> Parallel Hash (cost=8591.67..8591.67 rows=416667 width=4) (actual time=189.502..189.503 rows=333333 loops=3) Buckets: 131072 Batches: 16 Memory Usage: 3520kB -> Parallel Seq Scan on test1 t2 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.023..77.786 rows=333333 loops=3) Planning Time: 0.189 ms Execution Time: 565.448 ms (13 rows)
3.或者通过导入导出的方式,例如:
psql -h localhost -d postgres -U postgres -c "select count(*) from test t1,test1 t2 where t1.id = t2.id " -o result.csv -A -t -F "," psql -h localhost -d postgres -U postgres -c "COPY va FROM 'result.csv' WITH (FORMAT CSV, DELIMITER ',', HEADER FALSE, ENCODING 'windows-1252')"
一些场景下也会比非并行快。
到此这篇关于postgresql insert into select无法使用并行查询的解决的文章就介绍到这了,更多相关postgresql insert into select并行查询内容请搜索猪先飞以前的文章或继续浏览下面的相关文章希望大家以后多多支持猪先飞!
相关文章
- 这篇文章主要介绍了Mybatis Plus select 实现只查询部分字段的操作,具有很好的参考价值,希望对大家有所帮助。如有错误或未考虑完全的地方,望不吝赐教...2021-09-01
- 这篇文章主要介绍了PostgreSQL判断字符串是否包含目标字符串的多种方法,本文给大家介绍的非常详细,对大家的学习或工作具有一定的参考借鉴价值,需要的朋友可以参考下...2021-02-23
- 这篇文章主要介绍了解决Mybatis 大数据量的批量insert问题,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2021-01-09
- 这篇文章主要介绍了PostgreSQL TIMESTAMP类型 时间戳操作,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2020-12-26
- 1.在没有设置默认值的情况下: 复制代码 代码如下:SELECT userinfo.id, user_name, role, adm_regionid, region_name , create_timeFROM userinfoLEFT JOIN region ON userinfo.adm_regionid = region.id 结果:...2014-05-31
- 这篇文章主要介绍了vue 监听 Treeselect 选择项的改变操作,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2020-09-01
- 这篇文章主要介绍了postgresql 实现多表关联删除操作,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2021-01-02
- 这篇文章主要介绍了Postgresl 如何选择正确的关闭模式,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2021-01-18
使用insertAfter()方法在现有元素后添加一个新元素
复制代码 代码如下: //在现有元素后添加一个新元素 function insertAfter(newElement, targetElement){ var parent = targetElement.parentNode; if (parent.lastChild == targetElement){ parent.appendChild(newEl...2014-05-31vue Treeselect下拉树只能选择第N级元素实现代码
这篇文章主要介绍了vue Treeselect下拉树只能选择第N级元素实现代码,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2020-09-01- 这篇文章主要介绍了postgresql数据添加两个字段联合唯一的操作,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2021-02-04
- 这篇文章主要介绍了Vue select 绑定动态变量的实例讲解,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2020-10-23
- 这篇文章主要介绍了vue treeselect获取当前选中项的label实例,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2020-09-01
MySQL中SELECT+UPDATE处理并发更新问题解决方案分享
问题背景: 假设MySQL数据库有一张会员表vip_member(InnoDB表),结构如下: 当一个会员想续买会员(只能续买1个月、3个月或6个月)时,必须满足以下业务要求: •如果end_at早于当前时间,则设置start_at为当前时间,end_at为当前时...2014-05-31- 这篇文章主要介绍了PostgreSQL 字符串处理与日期处理操作,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2021-02-01
- 这篇文章主要介绍了postgresql重置序列起始值,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2021-01-04
- 下面小编就为大家带来一篇jQuery为动态生成的select元素添加事件的方法。小编觉得挺不错的,现在就分享给大家,也给大家做个参考。一起跟随小编过来看看吧...2016-09-01
- 这篇文章主要介绍javascript获取select标签选中的值方法,比较实用,需要的朋友可以参考下。...2016-06-12
- 这篇文章主要介绍了postgresql 中的时间处理小技巧(推荐),本文给大家介绍的非常详细,对大家的学习或工作具有一定的参考借鉴价值,需要的朋友可以参考下...2021-03-29
解决Mybatis中mapper.xml文件update,delete及insert返回值问题
这篇文章主要介绍了解决Mybatis中mapper.xml文件update,delete及insert返回值问题,具有很好的参考价值,希望对大家有所帮助。一起跟随小编过来看看吧...2020-11-23