support bloom-filter writing

[raphaelauv]

Feature Request / Improvement

Hi, py-iceberg cannot yet write bloom-filter, would be a great feature, thanks all

[ndrluis]

@sungwy I want to work on this. Can you assign it to me?

[ndrluis]

PyArrow does not support writing bloom filters. I opened an issue in the Arrow project apache/arrow#43530. I know that it's implemented in C++, but I don't know how easy it is to add it to the Python implementation. Therefore, I will try to contribute to the Arrow project.

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[raphaelauv]

No stale

[Fokko]

@raphaelauv Thanks, I've removed the label. The PR that we want to track is apache/arrow#37400. It is open since Aug 2023, but there is still activity. Hope it will land soon 🤞

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[raphaelauv]

no stale

[greenlaw]

Just stumbled across this issue and noticed that the reference pyarrow ticket has been merged as of ~2 weeks ago: apache/arrow#37400

so PyIceberg may be unblocked now.

[raulcd]

From my understanding the feature has been implemented on Arrow C++ but the capability hasn't been exposed to pyarrow yet. If someone wants to open an issue on the Arrow repository and work on the bindings for pyarrow I am happy to help on the review on the Arrow side. Otherwise unless someone starts working on it, I might try and work on it in the future but currently I am quite busy with other priorities.

[greenlaw]

@raulcd You're right, my mistake--the feature has only been merged in Arrow C++.

[renaudb]

Bloomfilter writing is now implemented in pyarrow and looks like it is set to be released in pyarrow 24 in a couple days. Is anyone looking into adding support to pyiceberg?

apache/arrow#49376
apache/arrow#49377