- 空間投影轉(zhuǎn)經(jīng)緯度坐標(biāo)系
- 測(cè)試數(shù)據(jù)文件類型片酝，shapefile
使用的中間函數(shù)有ST_PointFromText（將文本類的WKT數(shù)據(jù)文件轉(zhuǎn)成geometry類型的數(shù)據(jù)）
測(cè)試代碼如下：
import findspark
findspark.init()
from geospark.core.formatMapper.shapefileParser import ShapefileReader
from geospark.utils import KryoSerializer, GeoSparkKryoRegistrator
from pyspark import StorageLevel
from geospark.core.SpatialRDD import PointRDD
from geospark.core.enums import FileDataSplitter
from pyspark.sql import SparkSession
from geospark.register import GeoSparkRegistrator
from geospark.utils.adapter import Adapter
spark = SparkSession.builder\
.config("spark.serializer", KryoSerializer.getName)\
.config("spark.kryo.registrator", GeoSparkKryoRegistrator.getName).\
getOrCreate()
GeoSparkRegistrator.registerAll(spark)
input_location = r"D:\pycharm\pythonProject\GeoSpark\functions\taxi\txdata"
shapeRDD = ShapefileReader.readToGeometryRDD(spark.sparkContext,input_location)
df = Adapter.toDf(shapeRDD,spark)
df.show(truncate=False)
df.printSchema()
# shapeRDD.saveAsWKT("D:\pycharm\pythonProject\GeoSpark\functions\taxi\rsltWKT")
df.createOrReplaceTempView("p_view")
rsltDf = spark.sql(""" select *,ST_Transform(ST_PointFromText(p_view.geometry,"WKT"),\"epsg:3857\",\"epsg:4236\") from p_view """.strip())
rsltDf.show(truncate=False)
rsltDf.printSchema()
- 測(cè)試結(jié)果如下：
源數(shù)據(jù)文件打釉饕摹：
+---------------------------------------------+----+---------+---------+------+-------------------+
|geometry |OID_|ORIG_ID |DEST_ID |AMOUNT|DATE_TIME |
+---------------------------------------------+----+---------+---------+------+-------------------+
|POINT (-9301700.8034 5105596.222599998) |0 |258615475|437160417|8820 |2016-11-11 07:13:05|
|POINT (-9358740.9105 4037627.3462999985) |0 |704942357|510424955|4410 |2016-10-30 14:37:54|
|POINT (-9264386.5101 3682406.9626) |0 |001203959|404353450|8719 |2016-01-25 07:19:24|
|POINT (-9649006.4827 3951012.173100002) |0 |529097289|907207476|1372 |2016-10-23 17:35:57|
|POINT (-13180016.2029 4048140.6855999976) |0 |367009578|433332387|6182 |2016-06-12 16:46:49|
|POINT (-9630761.2182 4323872.919) |0 |400371343|874866791|6115 |2016-02-01 08:42:29|
|POINT (-8082997.2820999995 5188356.3825) |0 |969465806|897087904|9986 |2016-03-25 07:50:55|
|POINT (-8240558.8894 4888681.432899997) |0 |700460031|153246913|4916 |2016-02-20 07:12:31|
|POINT (-10854451.8527 4223728.012699999) |0 |651043044|676686033|2436 |2016-03-10 10:44:53|
|POINT (-9287006.6306 4606173.8500000015) |0 |464894120|465924488|8329 |2016-08-08 17:08:43|
|POINT (-8230317.496200001 4974816.362899996) |0 |369292057|829097328|7027 |2016-11-18 17:25:39|
|POINT (-13189033.0816 4072121.144299999) |0 |171982572|109247577|2613 |2016-09-03 09:30:19|
|POINT (-11851585.059500001 3734204.825000003)|0 |415102893|404170119|6796 |2016-12-30 14:42:47|
|POINT (-9248913.1009 5236898.5506) |0 |430608075|534452230|3408 |2016-02-09 17:50:48|
|POINT (-8422299.09 5071907.6696000025) |0 |543350607|469325922|8182 |2016-12-17 14:07:31|
|POINT (-8233445.573899999 4966253.920500003) |0 |400713444|423284456|1837 |2016-02-28 09:11:52|
|POINT (-9097474.0656 3551353.9376000017) |0 |405111054|400396185|8824 |2016-01-20 07:57:50|
|POINT (-13620574.219700001 5713090.735399999)|0 |543690234|691363014|3611 |2016-12-18 07:30:11|
|POINT (-10635297.1712 3459459.2875000015) |0 |162287502|989286406|1624 |2016-05-17 09:23:35|
|POINT (-13164542.7937 4030950.637599997) |0 |705176547|617526611|5482 |2016-09-23 07:48:18|
+---------------------------------------------+----+---------+---------+------+-------------------+
only showing top 20 rows
root
|-- geometry: string (nullable = true)
|-- OID_: string (nullable = true)
|-- ORIG_ID: string (nullable = true)
|-- DEST_ID: string (nullable = true)
|-- AMOUNT: string (nullable = true)
|-- DATE_TIME: string (nullable = true)
轉(zhuǎn)換后的數(shù)據(jù)打印：
+---------------------------------------------+----+---------+---------+------+-------------------+-------------------------------------------------------------------+
|geometry |OID_|ORIG_ID |DEST_ID |AMOUNT|DATE_TIME |st_transform(st_pointfromtext(geometry, WKT), epsg:3857, epsg:4236)|
+---------------------------------------------+----+---------+---------+------+-------------------+-------------------------------------------------------------------+
|POINT (-9301700.8034 5105596.222599998) |0 |258615475|437160417|8820 |2016-11-11 07:13:05|POINT (-83.55026400749416 41.63421823355603) |
|POINT (-9358740.9105 4037627.3462999985) |0 |704942357|510424955|4410 |2016-10-30 14:37:54|POINT (-84.06352152841099 34.070404819499196) |
|POINT (-9264386.5101 3682406.9626) |0 |001203959|404353450|8719 |2016-01-25 07:19:24|POINT (-83.2160684443583 31.386102079091344) |
|POINT (-9649006.4827 3951012.173100002) |0 |529097289|907207476|1372 |2016-10-23 17:35:57|POINT (-86.67132004895393 33.42352249150436) |
|POINT (-13180016.2029 4048140.6855999976) |0 |367009578|433332387|6182 |2016-06-12 16:46:49|POINT (-118.39485492578717 34.150156057427154) |
|POINT (-9630761.2182 4323872.919) |0 |400371343|874866791|6115 |2016-02-01 08:42:29|POINT (-86.50716135029704 36.17376931800233) |
|POINT (-8082997.2820999995 5188356.3825) |0 |969465806|897087904|9986 |2016-03-25 07:50:55|POINT (-72.60145509838317 42.18668862834412) |
|POINT (-8240558.8894 4888681.432899997) |0 |700460031|153246913|4916 |2016-02-20 07:12:31|POINT (-74.01723811036628 40.16055497253425) |
|POINT (-10854451.8527 4223728.012699999) |0 |651043044|676686033|2436 |2016-03-10 10:44:53|POINT (-97.50103376340422 35.44474772295976) |
|POINT (-9287006.6306 4606173.8500000015) |0 |464894120|465924488|8329 |2016-08-08 17:08:43|POINT (-83.41865851599233 38.19396835831678) |
|POINT (-8230317.496200001 4974816.362899996) |0 |369292057|829097328|7027 |2016-11-18 17:25:39|POINT (-73.92515285656063 40.74936267965346) |
|POINT (-13189033.0816 4072121.144299999) |0 |171982572|109247577|2613 |2016-09-03 09:30:19|POINT (-118.47586016147616 34.32827492016201) |
|POINT (-11851585.059500001 3734204.825000003)|0 |415102893|404170119|6796 |2016-12-30 14:42:47|POINT (-106.45979278427104 31.78365143151068) |
|POINT (-9248913.1009 5236898.5506) |0 |430608075|534452230|3408 |2016-02-09 17:50:48|POINT (-83.07590121227808 42.50981473259561) |
|POINT (-8422299.09 5071907.6696000025) |0 |543350607|469325922|8182 |2016-12-17 14:07:31|POINT (-75.6497927871166 41.407014403970145) |
|POINT (-8233445.573899999 4966253.920500003) |0 |400713444|423284456|1837 |2016-02-28 09:11:52|POINT (-73.95326278439005 40.69106344313047) |
|POINT (-9097474.0656 3551353.9376000017) |0 |405111054|400396185|8824 |2016-01-20 07:57:50|POINT (-81.71661953409682 30.375557412066826) |
|POINT (-13620574.219700001 5713090.735399999)|0 |543690234|691363014|3611 |2016-12-18 07:30:11|POINT (-122.35256948181386 45.585971687795634) |
|POINT (-10635297.1712 3459459.2875000015) |0 |162287502|989286406|1624 |2016-05-17 09:23:35|POINT (-95.53249791616715 29.661413022111155) |
|POINT (-13164542.7937 4030950.637599997) |0 |705176547|617526611|5482 |2016-09-23 07:48:18|POINT (-118.25583916701737 34.02224032714601) |
+---------------------------------------------+----+---------+---------+------+-------------------+-------------------------------------------------------------------+
- 在使用ST_PointFromText 注意傳入的類型。一般有TSV,CSV,WKT,WKB等等,
epsg:3857 是空間坐標(biāo)至非，epsg：4326經(jīng)緯度坐標(biāo)
