Changelog History
Page 3
-
v1.2.0 Changes
May 20, 2020๐ New Functionalities
- Infer mixed Parquet schemas on wr.s3.read_parquet_metadata and wr.s3.store_parquet_metadata #195
- โ Support to add new columns on wr.s3.to_parquet and wr.s3.store_parquet_metadata [TUTORIAL] #232
โจ Enhancements
- โ Now wr.s3.delete_objects raises exception for not deleted objects #237
- User-friendly exceptions on wr.athena.read_sql_query and wr.athena.read_sql_table #239
๐ Bug Fix
- Fix issue to use wr.s3.store_parquet_metadata on non-partitioned datasets #231
- โ Fix bug on wr.s3.read_json using chunksize #235
s3fs
version bumped #236- โ wr.s3.to_parquet single file does not sanitize column names fixed #240
Thanks
๐ We thank the following contributors/users for their work on this release:
@mrshu, @bryanyang0528, @JPFrancoia, @jaidisido, @qemtek, @dwbelliston, @mbiemann, @parasml, @BrainMonkey, @hyperloglog, @igorborgest.
_ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!
_ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).
-
v1.1.2 Changes
May 08, 2020๐ New Functionalities
- โ Add support for
uint8
,uint16
,uint32
anduint64
on Parquet. #76 - Add
get_table_parameters
,upsert_table_parameters
andupsert_table_parameters
onwr.catalog
. #224
โจ Enhancements
- โ Add readahead
cache
fors3fs
.
๐ Bug Fix
- ๐ Fixing type hints for sortkey. #226
- ๐ Fix
s3.to_parquet
overwriting with different partition schema.
Thanks
๐ We thank the following contributors/users for their work on this release:
@robertaves ,@jar-no1, @JPFrancoia, @igorborgest.
_ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!
_ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).
- โ Add support for
-
v1.1.1 Changes
May 06, 2020๐ Bug Fix
- Removing objects ending with "/" from
wr.s3.list_objects()
_ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!
_ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).
- Removing objects ending with "/" from
-
v1.1.0 Changes
May 05, 2020๐ New Functionalities
- ๐ Support for nested arrays and structs on
wr.s3.to_parquet()
#206 - ๐ Support for Read Parquet/Athena/Redshift chunked by number of rows #192
- Add
custom_classifications
towr.emr.create_cluster()
#193 - ๐ Support for Docker on EMR #193
- Add
kms_key_id
,max_file_size
,region
arguments towr.db.unload_redshift()
#197 - ๐ Add
catalog_versioning
argument towr.s3.to_csv()
andwr.s3.to_parquet()
#198 - Add
keep_files
andctas_temp_table_name
arguments towr.athena.read_sql_*()
#203 - Add
replace_filenames
argument towr.s3.copy_objects()
#215
โจ Enhancements
wr.s3.to_csv()
andwr.s3.to_parquet()
no longer need delete table permission to overwrite catalog table #198- Added support for UUID on
wr.db.read_sql_query()
(PostgreSQL) #200 - ๐จ Refactoring of Athena encryption and workgroup support #212
๐ Bug Fix
- ๐ Support for read full NULL columns from PostgreSQL, MySQL, and Redshift #218
Thanks
๐ We thank the following contributors/users for their work on this release:
@robkano ,@luigift, @parasml, @OElesin, @jar-no1, @keatmin, @pmleveque, @sapientderek, @jadayn, @igorborgest.
_ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!
_ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).
- ๐ Support for nested arrays and structs on
-
v1.0.4 Changes
April 20, 2020๐ New Functionalities
- โ Add wr.s3.copy_objects and wr.s3.merge_datasets #186
- Registering module's type annotations #194
โจ Enhancements
- Support
append
mode for wr.catalog.create_parquet_table and wr.catalog.create_csv_table #188
๐ Docs
- Adding a note about collisions for wr.catalog.sanitize_dataframe_columns_names #185
Thanks
๐ We thank the following contributors/users for their work on this release:
@JPFrancoia, @deathrowe, @igorborgest.
_ P.S. _ Lambda Layer's bundle and Glue's wheel/egg are available below. Just upload it and run!
_ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).
-
v1.0.3 Changes
April 15, 2020๐ New Functionalities
โจ Enhancements
- โ Add CSV tutorials #181
๐ Bug Fix
Thanks
๐ We thank the following contributors/users for their work on this release:
@russellbrooks, @vincentclaes, @JPFrancoia, @igorborgest.
_ P.S. _ Lambda Layer's bundle and Glue's wheel/egg are available below. Just upload it and run!
_ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).
-
v1.0.2 Changes
April 14, 2020๐ New Functionalities
โจ Enhancements
- Add
validate_schema
to wr.s3.to_parquet() #167
๐ Bug Fix
- โ Add CSV Dataset utilities to wr.s3.to_csv #170
- ๐ Fix CSV decompression #175
- ๐ Fix missing
boto3_session
#172
Thanks
๐ We thank the following contributors/users for their work on this release:
@vfrank66, @JPFrancoia, @jewelltp, @hjuhel-cdpq, @jar-no1, @rmlove, @josecw, @igorborgest.
_ P.S. _ Lambda Layer's bundle and Glue's wheel/egg are available below. Just upload it and run!
_ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).
- Add
-
v1.0.1 Changes
April 12, 2020๐ New Functionalities
- โ
categories
arg in s3.read_parquet, db.unload_redshift, athena.read_sql_query [#160]
โจ Enhancements
- Athena's table and columns names sanitisation revisited [#161]
๐ Bug Fix
- โ Add support for Athena queries on workgroups without encryption [#159]
Thanks
๐ We thank the following contributors/users for their work on this release:
@vfrank66, @nitin-kakkar, @sapientderek, @nagomiso, @igorborgest.
_ P.S. _ Lambda Layer's bundle and Glue's wheel/egg are available below. Just upload it and run!
_ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).
- โ
-
v1.0.0 Changes
April 10, 2020๐ฑ 1.0.0 ๐
๐ Check out the brand new documentation page!
_ P.S. _ Lambda Layer's bundle and Glue's wheel/egg are available below. Just upload it and run!
_ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).
-
v0.3.3
May 09, 2020