AWS Data Wrangler changelog

Changelog History

Page 3

• v1.2.0 Changes

  May 20, 2020

  🆕 New Functionalities

  • Infer mixed Parquet schemas on wr.s3.read_parquet_metadata and wr.s3.store_parquet_metadata #195
  • ✅ Support to add new columns on wr.s3.to_parquet and wr.s3.store_parquet_metadata [TUTORIAL] #232

  ✨ Enhancements

  • ✅ Now wr.s3.delete_objects raises exception for not deleted objects #237
  • User-friendly exceptions on wr.athena.read_sql_query and wr.athena.read_sql_table #239

  🐛 Bug Fix

  • Fix issue to use wr.s3.store_parquet_metadata on non-partitioned datasets #231
  • ✅ Fix bug on wr.s3.read_json using chunksize #235
  • s3fs version bumped #236
  • ✅ wr.s3.to_parquet single file does not sanitize column names fixed #240

  Thanks

  🚀 We thank the following contributors/users for their work on this release:

  @mrshu, @bryanyang0528, @JPFrancoia, @jaidisido, @qemtek, @dwbelliston, @mbiemann, @parasml, @BrainMonkey, @hyperloglog, @igorborgest.

  ---

  _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

  _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

• v1.1.2 Changes

  May 08, 2020

  🆕 New Functionalities

  • ➕ Add support for uint8, uint16, uint32 and uint64 on Parquet. #76
  • Add get_table_parameters, upsert_table_parameters and upsert_table_parameters on wr.catalog. #224

  ✨ Enhancements

  • ➕ Add readahead cache for s3fs.

  🐛 Bug Fix

  • 🛠 Fixing type hints for sortkey. #226
  • 🛠 Fix s3.to_parquet overwriting with different partition schema.

  Thanks

  🚀 We thank the following contributors/users for their work on this release:

  @robertaves ,@jar-no1, @JPFrancoia, @igorborgest.

  ---

  _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

  _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

• v1.1.1 Changes

  May 06, 2020

  🐛 Bug Fix

  • Removing objects ending with "/" from wr.s3.list_objects()

  ---

  _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

  _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

• v1.1.0 Changes

  May 05, 2020

  🆕 New Functionalities

  • 👌 Support for nested arrays and structs on wr.s3.to_parquet() #206
  • 👌 Support for Read Parquet/Athena/Redshift chunked by number of rows #192
  • Add custom_classifications to wr.emr.create_cluster() #193
  • 👌 Support for Docker on EMR #193
  • Add kms_key_id, max_file_size, region arguments to wr.db.unload_redshift() #197
  • 🔖 Add catalog_versioning argument to wr.s3.to_csv() and wr.s3.to_parquet() #198
  • Add keep_files and ctas_temp_table_name arguments to wr.athena.read_sql_*() #203
  • Add replace_filenames argument to wr.s3.copy_objects() #215

  ✨ Enhancements

  • wr.s3.to_csv() and wr.s3.to_parquet() no longer need delete table permission to overwrite catalog table #198
  • Added support for UUID on wr.db.read_sql_query()(PostgreSQL) #200
  • 🔨 Refactoring of Athena encryption and workgroup support #212

  🐛 Bug Fix

  • 👌 Support for read full NULL columns from PostgreSQL, MySQL, and Redshift #218

  Thanks

  🚀 We thank the following contributors/users for their work on this release:

  @robkano ,@luigift, @parasml, @OElesin, @jar-no1, @keatmin, @pmleveque, @sapientderek, @jadayn, @igorborgest.

  ---

  _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

  _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

• v1.0.4 Changes

  April 20, 2020

  🆕 New Functionalities

  • ✅ Add wr.s3.copy_objects and wr.s3.merge_datasets #186
  • Registering module's type annotations #194

  ✨ Enhancements

  • Support append mode for wr.catalog.create_parquet_table and wr.catalog.create_csv_table #188

  📄 Docs

  • Adding a note about collisions for wr.catalog.sanitize_dataframe_columns_names #185

  Thanks

  🚀 We thank the following contributors/users for their work on this release:

  @JPFrancoia, @deathrowe, @igorborgest.

  ---

  _ P.S. _ Lambda Layer's bundle and Glue's wheel/egg are available below. Just upload it and run!

  _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

• v1.0.3 Changes

  April 15, 2020

  🆕 New Functionalities

  • Add moto support for S3 and EMR (partially) #109

  ✨ Enhancements

  • ➕ Add CSV tutorials #181

  🐛 Bug Fix

  • Fix cast for char and varchar lengths #182
  • Fix Athena issues with boto3 session #179

  Thanks

  🚀 We thank the following contributors/users for their work on this release:

  @russellbrooks, @vincentclaes, @JPFrancoia, @igorborgest.

  ---

  _ P.S. _ Lambda Layer's bundle and Glue's wheel/egg are available below. Just upload it and run!

  _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

• v1.0.2 Changes

  April 14, 2020

  🆕 New Functionalities

  • Add new wr.catalog.extract_athena_types() #141

  ✨ Enhancements

  • Add validate_schema to wr.s3.to_parquet() #167

  🐛 Bug Fix

  • ✅ Add CSV Dataset utilities to wr.s3.to_csv #170
  • 🛠 Fix CSV decompression #175
  • 🛠 Fix missing boto3_session #172

  Thanks

  🚀 We thank the following contributors/users for their work on this release:

  @vfrank66, @JPFrancoia, @jewelltp, @hjuhel-cdpq, @jar-no1, @rmlove, @josecw, @igorborgest.

  ---

  _ P.S. _ Lambda Layer's bundle and Glue's wheel/egg are available below. Just upload it and run!

  _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

• v1.0.1 Changes

  April 12, 2020

  🆕 New Functionalities

  • ✅ categories arg in s3.read_parquet, db.unload_redshift, athena.read_sql_query [#160]

  ✨ Enhancements

  • Athena's table and columns names sanitisation revisited [#161]

  🐛 Bug Fix

  • ➕ Add support for Athena queries on workgroups without encryption [#159]

  Thanks

  🚀 We thank the following contributors/users for their work on this release:

  @vfrank66, @nitin-kakkar, @sapientderek, @nagomiso, @igorborgest.

  ---

  _ P.S. _ Lambda Layer's bundle and Glue's wheel/egg are available below. Just upload it and run!

  _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

• v1.0.0 Changes

  April 10, 2020

  🍱 1.0.0 🎉

  📚 Check out the brand new documentation page!

  ---

  _ P.S. _ Lambda Layer's bundle and Glue's wheel/egg are available below. Just upload it and run!

  _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

• v0.3.3

  May 09, 2020

• « First
• ‹ Prev
• 1
• 2
• 3
• 4
• Next ›
• Last »