All Versions
36
Latest Version
Avg Release Cycle
9 days
Latest Release
719 days ago

Changelog History
Page 2

  • v1.8.1 Changes

    August 11, 2020

    ๐Ÿ› Bug Fix

    • Fix NaN values handling for wr.athena.read_sql_*(). #351

    ๐Ÿ“„ Docs

    Thanks

    ๐Ÿš€ We thank the following contributors/users for their work on this release:

    @czagoni, @josecw, @igorborgest.


    _ P.S. _ Lambda Layer zip file and Glue wheel file are available below. Just upload it and run!

  • v1.8.0 Changes

    August 09, 2020

    ๐Ÿ†• New Functionalities

    • wr.s3.to_parquet() now has max_rows_by_file argument. #283
    • ๐Ÿ‘Œ Support for Unix path pattern matching (*, ?, [seq], [!seq]) for any list/read/delete/copy function on S3. #322

    โœจ Enhancements

    • Mypy applied with strict mode.

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix unnecessary table versioning (glue catalog) creation for wr.s3.to_parquet() during appends. #342
    • Lack of sanitisation in indexes names for wr.s3.to_parquet/csv(). #343

    ๐Ÿ“„ Docs

    Thanks

    ๐Ÿš€ We thank the following contributors/users for their work on this release:

    @Thiago-Dantas, @andre-marcos-perez, @ericct, @marcelo-vilela, @edvorkin, @nicholas-miles, @chrispruitt, @rparthas ,@igorborgest.


    _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

    _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

  • v1.7.0 Changes

    July 30, 2020

    ๐Ÿ’ฅ Breaking changes

    • The partitioned parquet reading now has a different approach for pushdown filters. For details check the tutorial

    ๐Ÿ†• New Functionalities

    โœจ Enhancements

    • ๐Ÿ‘Œ Support for PyArrow 1.0.0 #337
    • ๐Ÿ‘Œ Support for Pandas 1.1.0
    • ๐Ÿ‘Œ Support writing encrypted redshift copy manifest to S3 #327
    • wr.athane.read_sql_*() now accepts empty results #299
    • ๐Ÿ‘ Allow connect_args to be passed when creating an SQL engine from a glue connection #309
    • Add skip_header_line_count argument to wr.catalog.create_csv_table() #338

    ๐Ÿ› Bug Fix

    • โž• Add missing type annotations and fix types in docstrings. #321
    • KeyError: 'StatementType' with Athena using max_cache_seconds #323
    • wr.s3.read_csv() slow with chunksize #324
    • wr.s3.read_csv() with "chunksize" does not forward pandas_kwargs "encoding" #330
    • Ensure DataFrame mutability for wr.athane.read_sql_*() w/ ctas_approach=True #335

    ๐Ÿ“„ Docs

    • โšก๏ธ Several small updates.

    Thanks

    ๐Ÿš€ We thank the following contributors/users for their work on this release:

    @kylepierce, @davidszotten, @meganburger, @erikcw, @JPFrancoia, @zacharycarter, @DavideBossoli88, @c-line, @anand086, @jasadams, @mrtns, @schot, @koiker, @flaviomax, @bryanyang0528, @igorborgest.


    _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

    _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

  • v1.6.3 Changes

    July 12, 2020

    ๐Ÿ†• New Functionalities

    • โž• Add wr.catalog.get_partitions(). #305

    โœจ Enhancements

    • Improving Decimal casting.

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix support for support for boto3 >= 1.14.18. ๐Ÿž #315

    ๐Ÿ“„ Docs

    • โž• Add Spark Table Interoperability tutorial.
    • โšก๏ธ General small updates.

    Thanks

    ๐Ÿš€ We thank the following contributors/users for their work on this release:

    @jasadams, @bryanyang0528, @qemtek, @igorborgest.


    _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

    _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

  • v1.6.2 Changes

    July 01, 2020

    โœจ Enhancements

    • Now casting columns before append on an existing table only if necessary (wr.s3.to_parquet()).
    • โž• Add retry mechanism for InternalError on s3 object deletion.
    • โž• Add handling of immutable numpy arrays. (flag.writeable==False)

    _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

    _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

  • v1.6.1 Changes

    June 26, 2020

    โœจ Enhancements

    • Casting support for any column type to string using dtype argument on wr.s3.to_parquet()

    ๐Ÿ› Bug Fix

    • ๐Ÿฑ General bugs related to Athena Cache. ๐Ÿž

    ๐Ÿ“„ Docs

    • โšก๏ธ General small updates.

    _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

    _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

  • v1.6.0 Changes

    June 24, 2020

    ๐Ÿ†• New Functionalities

    • ๐Ÿฑ Amazon Athena CACHE ๐Ÿš€#285
    • ๐ŸŽ‰ Initial AWS STS module

    โœจ Enhancements

    • Numpy 1.19.0
    • Add auto_create and db_groups arguments to get_redshift_temp_engine #288
    • Add validate_schema arguments to wr.s3.read_parquet_table
    • โž• Add safe argument to read_parquet #296
    • ๐Ÿ”จ Refactor naming of pandas kwargs #291
    • Allow providing suffix to s3.store_parquet_metadata #295
    • Add last_modified_begin and last_modified_begin to list_objects, read_csv, read_json, read_fwf and read_parquet

    ๐Ÿ› Bug Fix

    • Fix bug on get_table_description on tables w/o description #294

    ๐Ÿ“„ Docs

    Thanks

    ๐Ÿš€ We thank the following contributors/users for their work on this release:

    @koiker, @patrick-muller, @flaviomax, @acere, @jarretg, @bryanyang0528, @schrobot, @kinghuang, @igorborgest.


    _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

    _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

  • v1.5.0 Changes

    June 14, 2020

    ๐Ÿ†• New Functionalities

    • ๐Ÿฑ Amazon QuickSight support! ๐ŸŽ‰
    • โž• Add create/delete database on wr.glue

    โœจ Enhancements

    • General improvements in the tutorials
    • ๐Ÿ†• New Amazon S3 path check
    • Add sanitize_columns arg for s3.to_parquet and s3.to_csv #278 #279
    • Remove memory copy of DataFrame for to_parquet and to_csv

    ๐Ÿ› Bug Fix

    • ๐Ÿ‘ฎ Force index=False for wr.db.to_sql() with redshift

    Thanks

    ๐Ÿš€ We thank the following contributors/users for their work on this release:

    @ywang103, @patrick-muller, @tuliocasagrande, @sarojdongol, @sdknij, @ilyanoskov, @igorborgest.


    _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

    _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

  • v1.4.0 Changes

    June 02, 2020

    ๐Ÿ†• New Functionalities

    • โž• Add support for reading CSV, JSON and FWF partitions. #265

    โœจ Enhancements

    • โœ… General improvement of moto tests

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix encoding arg support for reading CSV, JSON and FWF. #271

    Thanks

    ๐Ÿš€ We thank the following contributors/users for their work on this release:

    @bryanyang0528, @dwbelliston, @patrick-muller, @sdknij, @igorborgest.


    _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

    _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).

  • v1.3.0 Changes

    May 28, 2020

    ๐Ÿ†• New Functionalities

    • ๐Ÿ‘Œ Support for Athena Partition Projection [TUTORIAL]

    โœจ Enhancements

    • โฌ†๏ธ Bumping SQLAlchemy version to 1.3.15 #259
    • โœ… General improvement of moto tests #254

    ๐Ÿ› Bug Fix

    • ๐Ÿ›  Fix dtype (cast) on wr.s3.to_parquet with nested types #263
    • ๐Ÿ›  Fix EMR utilities for others region different than us-east-1 #252
    • ๐Ÿ›  Fix wr.s3.to_parquet for partitions in reverse order #264

    Thanks

    ๐Ÿš€ We thank the following contributors/users for their work on this release:

    @bryanyang0528, @zachmoshe, @buseynehannes, @jiajie999, @igorborgest.


    _ P.S. _ Lambda Layer's zip-file and Glue's wheel/egg are available below. Just upload it and run!

    _ P.P.S. _ AWS Data Wrangler counts on compiled dependencies (C/C++) so there is no support for Glue PySpark by now (Only Glue Python Shell).