Bigbench tensorflow GPU dependency

Something isn't working

Thanks for reporting ! :) cc @andersjohanandreassen can you take a look at this ?

Also @cceyda feel free to open an issue at [BIG-Bench](https://github.com/google/BIG-bench) as well regarding the `AttributeError`

I'm on vacation for the next week, so won't be able to do much debugging at the moment. Sorry for the inconvenience.
But I did quickly take a look:

**pypi**:
I managed to reproduce the above error with the pypi version begin out of date.
The version on `https://storage.googleapis.com/public_research_data/bigbench/bigbench-0.0.1.tar.gz` should be up to date, but it was my understanding that there was some issue with the pypi upload, so I don't even understand why there is a version [on pypi from April 1](https://pypi.org/project/bigbench/0.0.1/). Perhaps @ethansdyer, who's handling the pypi upload, knows the answer to that?

**OOM error**:
But, I'm unable to reproduce the OOM error in a google colab with GPU enabled.
This is what I ran:
```
!pip install bigbench@https://storage.googleapis.com/public_research_data/bigbench/bigbench-0.0.1.tar.gz
!pip install datasets

from datasets import load_dataset
dataset = load_dataset("bigbench","swedish_to_german_proverbs")
```
The `swedish_to_german_proverbs`task is only 72 examples, so I don't understand what could be causing the OOM error. Loading the task has no effect on the RAM for me. @cceyda Can you confirm that this does not occur in a [colab](https://colab.research.google.com/)?
If the GPU is somehow causing issues on your system, disabling the GPU from TF might be an option too
```
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"
```

Solved.
Yes it works on colab, and somehow magically on my machine too now. hmm not sure what was wrong before I had used a fresh venv both times with just the dataloading code, and tried multiple times. (maybe just a wrong tensorflow version got mixed up somehow) The tensorflow call seems to come from the bigbench side anyway.

about bigbench pypi version update, I opened an issue over there https://github.com/google/BIG-bench/issues/846

anyway closing this now. If anyone else has the same problem can re-open.

## Describe the bug
Loading bigbech
```py
from datasets import load_dataset
dataset = load_dataset("bigbench","swedish_to_german_proverbs")
```
tries to use gpu and fails with OOM with the following error

```
Downloading and preparing dataset bigbench/swedish_to_german_proverbs (download: Unknown size, generated: 68.92 KiB, post-processed: Unknown size, total: 68.92 KiB) to /home/ceyda/.cache/huggingface/datasets/bigbench/swedish_to_german_proverbs/1.0.0/7d2f6e537fa937dfaac8b1c1df782f2055071d3fd8e4f4ae93d28012a354ced0...
Generating default split:   0%|                                                                                | 0/72 [00:00<?, ? examples/s]2022-06-13 14:11:04.154469: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-06-13 14:11:05.133600: F tensorflow/core/platform/statusor.cc:33] Attempting to fetch value instead of handling error INTERNAL: failed initializing StreamExecutor for CUDA device ordinal 3: INTERNAL: failed call to cuDevicePrimaryCtxRetain: CUDA_ERROR_OUT_OF_MEMORY: out of memory; total memory reported: 25396838400
Aborted (core dumped)
```
I think this is because bigbench dependency (below) installs tensorflow (GPU version) and dataloading tries to use GPU as default.

`pip install bigbench@https://storage.googleapis.com/public_research_data/bigbench/bigbench-0.0.1.tar.gz`

while just doing 'pip install bigbench' results in following error
```
File "/home/ceyda/.local/lib/python3.7/site-packages/datasets/load.py", line 109, in import_main_class
    module = importlib.import_module(module_path)
  File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/ceyda/.cache/huggingface/modules/datasets_modules/datasets/bigbench/7d2f6e537fa937dfaac8b1c1df782f2055071d3fd8e4f4ae93d28012a354ced0/bigbench.py", line 118, in <module>
    class Bigbench(datasets.GeneratorBasedBuilder):
  File "/home/ceyda/.cache/huggingface/modules/datasets_modules/datasets/bigbench/7d2f6e537fa937dfaac8b1c1df782f2055071d3fd8e4f4ae93d28012a354ced0/bigbench.py", line 127, in Bigbench
    BigBenchConfig(name=name, version=datasets.Version("1.0.0")) for name in bb_utils.get_all_json_task_names()
AttributeError: module 'bigbench.api.util' has no attribute 'get_all_json_task_names'
```

## Steps to avoid the bug
Not ideal but can solve with (since I don't really use tensorflow elsewhere)
`pip uninstall tensorflow`
`pip install tensorflow-cpu`

## Environment info
- datasets @ master
- Python version: 3.7


Include entity positions as feature in ReCoRD

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_4479). All of your documentation changes will be reflected on that endpoint.

https://huggingface.co/datasets/super_glue/viewer/record/validation

TLDR: We need to record entity positions, which are included in the source data but excluded by the loading script, to enable efficient and effective training for ReCoRD.

Currently, the loading script ignores the entity positions ("entity_start", "entity_end") and only records entity text. This might be because the training method of the official baseline is to make n training instance from a datapoint by replacing \"\@ placeholder\" in query with each entity individually.

But it increases the already heavy computation by multiple folds. So DeBERTa uses a method that take entity embeddings by their positions in the passage, and thus makes one training instance from one data point. It is way more efficient and proved effective for the ReCoRD task.

Can anybody help me with the dataset card rendering error? Maybe @lhoestq ?


Dataset slow during model training

bug

Something isn't working

Hi ! cc @Rocketknight1 maybe you know better ?

I'm not too familiar with `tf.data.experimental.save`. Note that `datasets` uses memory mapping, so depending on your hardware and the disk you are using you can expect performance differences with a dataset loaded in RAM

Hi @lehrig, I suspect what's happening here is that our `to_tf_dataset()` method has some performance issues when streaming samples. This is usually not a problem, but they become apparent when streaming a vision dataset into a very small vision model, which will need a lot of sample throughput to saturate the GPU.

When you save a `tf.data.Dataset` with `tf.data.experimental.save`, all of the samples from the dataset (which are, in this case, batches of images), are saved to disk. When you load this saved dataset, you're effectively bypassing `to_tf_dataset()` entirely, which alleviates this performance bottleneck.

`to_tf_dataset()` is something we're actively working on overhauling right now - particularly for image datasets, we want to make it possible to access the underlying images with `tf.data` without going through the current layer of indirection with `Arrow`, which should massively improve simplicity and performance.

However, if you just want this to work quickly but without needing your save/load hack, my advice would be to simply load the dataset into memory if it's small enough to fit. Since all your samples have the same dimensions, you can do this simply with:

```
dataset = load_from_disk(prep_data_dir)
dataset = dataset.with_format("numpy")
data_in_memory = dataset[:]
```

Then you can simply do something like:

```
model.fit(data_in_memory["pixel_values"], data_in_memory["labels"])
```

Thanks for the information!

I have now updated the training code like so:

```
dataset = load_from_disk(prep_data_dir)
train_dataset = dataset["train"][:]
validation_dataset = dataset["dev"][:]

...

model.fit(
    train_dataset["pixel_values"],
    train_dataset["label"],
    epochs=epochs,
    validation_data=(
        validation_dataset["pixel_values"],
        validation_dataset["label"]
    ),
    callbacks=[earlyStopping, mcp_save, reduce_lr_loss]
)
```

- Creating the in-memory dataset is quite quick
- But: There is now a long wait (~4-5 Minutes) before the training starts (why?)
- And: Training times have improved but the very first epoch leaves me wondering why it takes so long (why?)

**Epoch Breakdown:**
- Epoch 1/10
78s 12s/step - loss: 3.1307 - accuracy: 0.0737 - val_loss: 2.2827 - val_accuracy: 0.1273 - lr: 0.0010
- Epoch 2/10
1s 168ms/step - loss: 2.3616 - accuracy: 0.2350 - val_loss: 2.2679 - val_accuracy: 0.2182 - lr: 0.0010
- Epoch 3/10
1s 189ms/step - loss: 2.0221 - accuracy: 0.3180 - val_loss: 2.2670 - val_accuracy: 0.1818 - lr: 0.0010
- Epoch 4/10
0s 67ms/step - loss: 1.8895 - accuracy: 0.3548 - val_loss: 2.2771 - val_accuracy: 0.1273 - lr: 0.0010
- Epoch 5/10
0s 67ms/step - loss: 1.7846 - accuracy: 0.3963 - val_loss: 2.2860 - val_accuracy: 0.1455 - lr: 0.0010
- Epoch 6/10
0s 65ms/step - loss: 1.5946 - accuracy: 0.4516 - val_loss: 2.2938 - val_accuracy: 0.1636 - lr: 0.0010
- Epoch 7/10
0s 63ms/step - loss: 1.4217 - accuracy: 0.5115 - val_loss: 2.2968 - val_accuracy: 0.2182 - lr: 0.0010
- Epoch 8/10
0s 67ms/step - loss: 1.3089 - accuracy: 0.5438 - val_loss: 2.2842 - val_accuracy: 0.2182 - lr: 0.0010
- Epoch 9/10
1s 184ms/step - loss: 1.2480 - accuracy: 0.5806 - val_loss: 2.2652 - val_accuracy: 0.1818 - lr: 0.0010
- Epoch 10/10
0s 65ms/step - loss: 1.2699 - accuracy: 0.5622 - val_loss: 2.2670 - val_accuracy: 0.2000 - lr: 0.0010

Regarding the new long ~5 min. wait introduced by the in-memory dataset update: this might be causing it? https://datascience.stackexchange.com/questions/33364/why-model-fit-generator-in-keras-is-taking-so-much-time-even-before-picking-the

For now, my save/load hack is still more performant, even though having more boiler-plate code :/

That 5 minute wait is quite surprising! I don't have a good explanation for why it's happening, but it can't be an issue with `datasets` or `tf.data` because you're just fitting directly on Numpy arrays at this point. All I can suggest is seeing if you can isolate the issue - for example, does fitting on a smaller dataset containing only 10% of the original data reduce the wait? This might indicate the delay is caused by your data being copied or converted somehow. Alternatively, you could try removing things like callbacks and seeing if you could isolate the issue there.

## Describe the bug
While migrating towards 🤗 Datasets, I encountered an odd performance degradation: training suddenly slows down dramatically. I train with an image dataset using Keras and execute a `to_tf_dataset` just before training.

First, I have optimized my dataset following https://discuss.huggingface.co/t/solved-image-dataset-seems-slow-for-larger-image-size/10960/6, which actually improved the situation from what I had before but did not completely solve it.

Second, I saved and loaded my dataset using `tf.data.experimental.save` and `tf.data.experimental.load` before training (for which I would have expected no performance change). However, I ended up with the performance I had before tinkering with 🤗 Datasets.

Any idea what's the reason for this and how to speed-up training with 🤗 Datasets?

## Steps to reproduce the bug
```python
# Sample code to reproduce the bug

from datasets import load_dataset
import os

dataset_dir = "./dataset"
prep_dataset_dir = "./prepdataset"
model_dir = "./model"

# Load Data
dataset = load_dataset("Lehrig/Monkey-Species-Collection", "downsized")
def read_image_file(example):
    with open(example["image"].filename, "rb") as f:
        example["image"] = {"bytes": f.read()}
        return example
dataset = dataset.map(read_image_file)
dataset.save_to_disk(dataset_dir)

# Preprocess
from datasets import (
    Array3D,
    DatasetDict,
    Features,
    load_from_disk,
    Sequence,
    Value
)
import numpy as np
from transformers import ImageFeatureExtractionMixin

dataset = load_from_disk(dataset_dir)

num_classes = dataset["train"].features["label"].num_classes
one_hot_matrix = np.eye(num_classes)
feature_extractor = ImageFeatureExtractionMixin()

def to_pixels(image):
    image = feature_extractor.resize(image, size=size)
    image = feature_extractor.to_numpy_array(image, channel_first=False)
    image = image / 255.0
    return image

def process(examples):
    examples["pixel_values"] = [
        to_pixels(image) for image in examples["image"]
    ]
    examples["label"] = [
        one_hot_matrix[label] for label in examples["label"]
    ]
    return examples

features = Features({
    "pixel_values": Array3D(dtype="float32", shape=(size, size, 3)),
    "label": Sequence(feature=Value(dtype="int32"), length=num_classes)
})

prep_dataset = dataset.map(
    process,
    remove_columns=["image"],
    batched=True,
    batch_size=batch_size,
    num_proc=2,
    features=features,
)

prep_dataset = prep_dataset.with_format("numpy")

# Split
train_dev_dataset = prep_dataset['test'].train_test_split(
    test_size=test_size,
    shuffle=True,
    seed=seed
)

train_dev_test_dataset = DatasetDict({
    'train': train_dev_dataset['train'],
    'dev': train_dev_dataset['test'],
    'test': prep_dataset['test'],
})

train_dev_test_dataset.save_to_disk(prep_dataset_dir)

# Train Model
import datetime
import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.applications import InceptionV3
from tensorflow.keras.layers import Dense, Dropout, GlobalAveragePooling2D, BatchNormalization
from tensorflow.keras.callbacks import ReduceLROnPlateau, ModelCheckpoint, EarlyStopping
from transformers import DefaultDataCollator

dataset = load_from_disk(prep_data_dir)

data_collator = DefaultDataCollator(return_tensors="tf")

train_dataset = dataset["train"].to_tf_dataset(
    columns=['pixel_values'],
    label_cols=['label'],
    shuffle=True,
    batch_size=batch_size,
    collate_fn=data_collator
)

validation_dataset = dataset["dev"].to_tf_dataset(
    columns=['pixel_values'],
    label_cols=['label'],
    shuffle=False,
    batch_size=batch_size,
    collate_fn=data_collator
)

print(f'{datetime.datetime.now()} - Saving Data')
tf.data.experimental.save(train_dataset, model_dir+"/train")
tf.data.experimental.save(validation_dataset, model_dir+"/val")

print(f'{datetime.datetime.now()} - Loading Data')
train_dataset = tf.data.experimental.load(model_dir+"/train")
validation_dataset = tf.data.experimental.load(model_dir+"/val")

shape = np.shape(dataset["train"][0]["pixel_values"])
backbone = InceptionV3(
    include_top=False,
    weights='imagenet',
    input_shape=shape
)

for layer in backbone.layers:
    layer.trainable = False

model = Sequential()
model.add(backbone)
model.add(GlobalAveragePooling2D())
model.add(Dense(128, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.3))
model.add(Dense(64, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.3))
model.add(Dense(10, activation='softmax'))

model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)

print(model.summary())

earlyStopping = EarlyStopping(
    monitor='val_loss',
    patience=10,
    verbose=0,
    mode='min'
)

mcp_save = ModelCheckpoint(
    f'{model_dir}/best_model.hdf5',
    save_best_only=True,
    monitor='val_loss',
    mode='min'
)

reduce_lr_loss = ReduceLROnPlateau(
    monitor='val_loss',
    factor=0.1,
    patience=7,
    verbose=1,
    min_delta=0.0001,
    mode='min'
)

hist = model.fit(
    train_dataset,
    epochs=epochs,
    validation_data=validation_dataset,
    callbacks=[earlyStopping, mcp_save, reduce_lr_loss]
)
```

## Expected results
Same performance when training without my "save/load hack" or a good explanation/recommendation about the issue.

## Actual results
Performance slower without my "save/load hack".

**Epoch Breakdown (without my "save/load hack"):**
- Epoch 1/10
41s 2s/step - loss: 1.6302 - accuracy: 0.5048 - val_loss: 1.4713 - val_accuracy: 0.3273 - lr: 0.0010
- Epoch 2/10
32s 2s/step - loss: 0.5357 - accuracy: 0.8510 - val_loss: 1.0447 - val_accuracy: 0.5818 - lr: 0.0010
- Epoch 3/10
36s 3s/step - loss: 0.3547 - accuracy: 0.9231 - val_loss: 0.6245 - val_accuracy: 0.7091 - lr: 0.0010
- Epoch 4/10
36s 3s/step - loss: 0.2721 - accuracy: 0.9231 - val_loss: 0.3395 - val_accuracy: 0.9091 - lr: 0.0010
- Epoch 5/10
32s 2s/step - loss: 0.1676 - accuracy: 0.9856 - val_loss: 0.2187 - val_accuracy: 0.9636 - lr: 0.0010
- Epoch 6/10
42s 3s/step - loss: 0.2066 - accuracy: 0.9615 - val_loss: 0.1635 - val_accuracy: 0.9636 - lr: 0.0010
- Epoch 7/10
32s 2s/step - loss: 0.1814 - accuracy: 0.9423 - val_loss: 0.1418 - val_accuracy: 0.9636 - lr: 0.0010
- Epoch 8/10
32s 2s/step - loss: 0.1301 - accuracy: 0.9856 - val_loss: 0.1388 - val_accuracy: 0.9818 - lr: 0.0010
- Epoch 9/10
loss: 0.1102 - accuracy: 0.9856 - val_loss: 0.1185 - val_accuracy: 0.9818 - lr: 0.0010
- Epoch 10/10
32s 2s/step - loss: 0.1013 - accuracy: 0.9808 - val_loss: 0.0978 - val_accuracy: 0.9818 - lr: 0.0010

**Epoch Breakdown (with my "save/load hack"):**
- Epoch 1/10
13s 625ms/step - loss: 3.0478 - accuracy: 0.1146 - val_loss: 2.3061 - val_accuracy: 0.0727 - lr: 0.0010
- Epoch 2/10
0s 80ms/step - loss: 2.3105 - accuracy: 0.2656 - val_loss: 2.3085 - val_accuracy: 0.0909 - lr: 0.0010
- Epoch 3/10
0s 77ms/step - loss: 1.8608 - accuracy: 0.3542 - val_loss: 2.3130 - val_accuracy: 0.0909 - lr: 0.0010
- Epoch 4/10
1s 98ms/step - loss: 1.8677 - accuracy: 0.3750 - val_loss: 2.3157 - val_accuracy: 0.0909 - lr: 0.0010
- Epoch 5/10
1s 204ms/step - loss: 1.5561 - accuracy: 0.4583 - val_loss: 2.3049 - val_accuracy: 0.0909 - lr: 0.0010
- Epoch 6/10
1s 210ms/step - loss: 1.4657 - accuracy: 0.4896 - val_loss: 2.2944 - val_accuracy: 0.0909 - lr: 0.0010
- Epoch 7/10
1s 205ms/step - loss: 1.4018 - accuracy: 0.5312 - val_loss: 2.2917 - val_accuracy: 0.0909 - lr: 0.0010
- Epoch 8/10
1s 207ms/step - loss: 1.2370 - accuracy: 0.5729 - val_loss: 2.2814 - val_accuracy: 0.0909 - lr: 0.0010
- Epoch 9/10
1s 214ms/step - loss: 1.1190 - accuracy: 0.6250 - val_loss: 2.2733 - val_accuracy: 0.0909 - lr: 0.0010
- Epoch 10/10
1s 207ms/step - loss: 1.1484 - accuracy: 0.6302 - val_loss: 2.2624 - val_accuracy: 0.0909 - lr: 0.0010

## Environment info
- `datasets` version: 2.2.2
- Platform: Linux-4.18.0-305.45.1.el8_4.ppc64le-ppc64le-with-glibc2.17
- Python version: 3.8.13
- PyArrow version: 7.0.0
- Pandas version: 1.4.2
- TensorFlow: 2.8.0
- GPU (used during training): Tesla V100-SXM2-32GB


Dataset Viewer issue for fgrezes/WIESP2022-NER

https://huggingface.co/datasets/fgrezes/WIESP2022-NER

The error:

```
Message:       Couldn't find a dataset script at /src/services/worker/fgrezes/WIESP2022-NER/WIESP2022-NER.py or any data file in the same directory. Couldn't find 'fgrezes/WIESP2022-NER' on the Hugging Face Hub either: FileNotFoundError: Unable to resolve any data file that matches ['**test*', '**eval*'] in dataset repository fgrezes/WIESP2022-NER with any supported extension ['csv', 'tsv', 'json', 'jsonl', 'parquet', 'txt', 'blp', 'bmp', 'dib', 'bufr', 'cur', 'pcx', 'dcx', 'dds', 'ps', 'eps', 'fit', 'fits', 'fli', 'flc', 'ftc', 'ftu', 'gbr', 'gif', 'grib', 'h5', 'hdf', 'png', 'apng', 'jp2', 'j2k', 'jpc', 'jpf', 'jpx', 'j2c', 'icns', 'ico', 'im', 'iim', 'tif', 'tiff', 'jfif', 'jpe', 'jpg', 'jpeg', 'mpg', 'mpeg', 'msp', 'pcd', 'pxr', 'pbm', 'pgm', 'ppm', 'pnm', 'psd', 'bw', 'rgb', 'rgba', 'sgi', 'ras', 'tga', 'icb', 'vda', 'vst', 'webp', 'wmf', 'emf', 'xbm', 'xpm', 'zip']
```

I understand the issue is not related to the dataset viewer in itself, but with the autodetection of the data files without a loading script in the datasets library. cc @lhoestq @albertvillanova @mariosasko

Apparently it finds `scoring-scripts/compute_seqeval.py` which matches `**eval*`, a regex that detects a test split. We should probably improve the regex because it's not supposed to catch this kind of files. It must also only check for files with supported extensions: txt, csv, png etc.

### Link

_No response_

### Description

_No response_

### Owner

_No response_


`to_pandas` doesn't take into account format.

enhancement

New feature or request

Thanks for opening a discussion :)

Note that you can use `.remove_columns(...)` to keep only the ones you're interested in before calling `.to_pandas()`

Yes I can do that thank you!

Do you think that conceptually my example should work? If not, I'm happy to close this issue.

If yes, I can start working on it.

Hi! Instead of `with_format(columns=['a', 'b']).to_pandas()`, use `with_format("pandas", columns=["a", "b"])` for easy conversion of the parts of the dataset to pandas via indexing/slicing.

The full code:
```python
from datasets import Dataset

ds = Dataset.from_dict({'a': [1,2,3], 'b': [5,6,7], 'c': [8,9,10]})
pandas_df = ds.with_format("pandas", columns=['a', 'b'])[:]
```

**Is your feature request related to a problem? Please describe.**

I have a large dataset that I need to convert part of to pandas to do some further analysis. Calling `to_pandas` directly on it is expensive. So I thought I could simply select the columns that I want and then call `to_pandas`.

**Describe the solution you'd like**

```python
from datasets import Dataset

ds = Dataset.from_dict({'a': [1,2,3], 'b': [5,6,7], 'c': [8,9,10]})
pandas_df = ds.with_format(columns=['a', 'b']).to_pandas()

# I would expect `pandas_df` to only include a,b as column.
```

**Describe alternatives you've considered**
I could remove all columns that I don't want? But I don't know all of them in advance.

**Additional context**
I can probably make a PR with some pointers.


Improve error message for missing pacakges from inside dataset script

_The documentation is not available anymore as the PR was closed or merged._

I opened a PR before I noticed yours ^^' You can find it here: https://github.com/huggingface/datasets/pull/4484

The only comment I have regarding your message is that it possibly shows several `pip install` commands, whereas one can run one single `pip install` command with the list of missing dependencies, which is maybe simpler.

Let me know which one your prefer

Improve the error message for missing packages from inside a dataset script:

With this change, the error message for missing packages for `bigbench` looks as follows:
```
ImportError: To be able to use bigbench, you need to install the following dependencies:
        - 'bigbench' using 'pip install "bigbench @ https://storage.googleapis.com/public_research_data/bigbench/bigbench-0.0.1.tar.gz"'
```

And this is how it looked before:
```
ImportError: To be able to use bigbench, you need to install the following dependencies['bigbench', 'bigbench', 'bigbench', 'bigbench'] using 'pip install "bigbench @ https://storage.googleapis.com/public_research_data/bigbench/bigbench-0.0.1.tar.gz" bigbench bigbench bigbench' for instance'
```


[Docs] How to use with PyTorch page

documentation

Improvements or additions to documentation

Currently the docs about PyTorch are scattered around different pages, and we were missing a place to explain more in depth how to use and optimize a dataset for PyTorch. This PR is related to #4457 which is the TF counterpart :)

cc @Rocketknight1 we can try to align both documentations contents now I think

cc @stevhliu let me know what you think !


Add SST-2 dataset

on the hub this dataset is referenced as `sst-2` not `sst2` – is there a canonical orthography? If not, could we name it `sst-2`?

@julien-c, we normally do not use hyphens for dataset names: whenever the original dataset name contains a hyphen, we usually:
- either suppress it: CoNLL-2000 (`conll2000`), CORD-19 (`cord19`)
- or replace it with underscore: CC-News (`cc_news`), SQuAD-es (`squad_es`)

There are some exceptions though... (I wonder why)

I think, the reason is there was a 1-to-1 relation with the corresponding Python module name.

I personally find confusing not having a rule and using both hyphens and underscores indistinctly: you never know which is the right orthography.

Whichever the decision we make, I would prefer to be applied consistently.

Also note that we already implemented this dataset as part of GLUE: https://github.com/huggingface/datasets/blob/master/datasets/glue/glue.py#L163
- dataset name: `glue`
- config name: `sst2`

On the other hand, let's see how other libraries name it:
- torchtext: `SST2` https://pytorch.org/text/stable/datasets.html#sst2
- OpenAI CLIP: `rendered-sst2` https://github.com/openai/CLIP/blob/main/data/rendered-sst2.md
- Kaggle: `SST2` https://www.kaggle.com/datasets/atulanandjha/stanford-sentiment-treebank-v2-sst2/version/22
- TensorFlow Datasets: `glue/sst2` https://www.tensorflow.org/datasets/catalog/glue#gluesst2

Ok, another option is to open PRs against the models in https://huggingface.co/models?datasets=sst-2 to change their dataset reference to `sst2`

(BTW some models refer to `sst2` already – but they're less popular: https://huggingface.co/models?datasets=sst2)

OK, I'm taking care of the subsequent PRs on models to align with this dataset name.

Add SST-2 dataset.

Currently it is part of GLUE benchmark.

This PR adds it as a standalone dataset.

CC: @julien-c


Fix 401 error for unauthticated requests to non-existing repos

The hub now returns 401 instead of 404 for unauthenticated requests to non-existing repos.
This PR add support for the 401 error and fixes the CI fails on `master`


CI error with repo lhoestq/_dummy

bug

Something isn't working

## Describe the bug
CI is failing because of repo "lhoestq/_dummy". See: https://app.circleci.com/pipelines/github/huggingface/datasets/12461/workflows/1b040b45-9578-4ab9-8c44-c643c4eb8691/jobs/74269
```
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/api/datasets/lhoestq/_dummy?full=true
```

The repo seems to no longer exist: https://huggingface.co/api/datasets/lhoestq/_dummy
```
error: "Repository not found"
```

CC: @lhoestq


Reorder returned validation/test splits in script template


Replace data URLs in wider_face dataset once hosted on the Hub

This PR replaces the URLs of data files in Google Drive with our Hub ones, once the data owners have approved to host their data on the Hub.

They also informed us that their dataset is licensed under CC BY-NC-ND.


Generalize tutorials for audio and vision

This PR updates the tutorials to be more generalizable to all modalities. After reading the tutorials, a user should be able to load any type of dataset, know how to index into and slice a dataset, and do the most basic/common type of preprocessing (tokenization, resampling, applying transforms) depending on their dataset.

Other changes include:

- Removed the sections about a dataset's metadata, features, and columns because we cover this in an earlier tutorial about inspecting the `DatasetInfo` through the dataset builder.
- Separate the sharing dataset tutorial into two sections: (1) uploading via the web interface and (2) using the `huggingface_hub` library.
- Renamed some tutorials in the TOC to be more clear and specific.
- Added more text to nudge users towards joining the community and asking questions on the forums.
- If it's okay with everyone, I'd also like to remove the section about loading and using metrics since we have the `evaluate` docs now.


Transcript string 'null' converted to [None] by load_dataset()

Something isn't working

Hi @mbarnig, thanks for reporting.

Please note that is an expected behavior by `pandas` (we use the `pandas` library to parse CSV files): https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html
```
By default the following values are interpreted as NaN:
‘’, ‘#N/A’, ‘#N/A N/A’, ‘#NA’, ‘-1.#IND’, ‘-1.#QNAN’, ‘-NaN’, ‘-nan’, ‘1.#IND’, ‘1.#QNAN’, ‘<NA>’, ‘N/A’, ‘NA’, ‘NULL’, ‘NaN’, ‘n/a’, ‘nan’, ‘null’.
```
(see "null" in the last position in the above list).

In order to prevent `pandas` from performing that automatic conversion from the string "null" to a NaN value, you should pass the `pandas` parameter `keep_default_na=False`:
```python
In [2]: dataset = load_dataset('csv', data_files={'train': 'null-test.csv'}, keep_default_na=False)
In [3]: dataset["train"][0]["transcript"]
Out[3]: 'null'
```

## Issue
I am training a luxembourgish speech-recognition model in Colab with a custom dataset, including a dictionary of luxembourgish words, for example the speaken numbers 0 to 9. When preparing the dataset with the script

`ds_train1 = mydataset.map(prepare_dataset)`

the following error was issued:

```
ValueError                                Traceback (most recent call last)
<ipython-input-69-1e8f2b37f5bc> in <module>()
----> 1 ds_train = mydataset_train.map(prepare_dataset)

11 frames
/usr/local/lib/python3.7/dist-packages/transformers/tokenization_utils_base.py in __call__(self, text, text_pair, add_special_tokens, padding, truncation, max_length, stride, is_split_into_words, pad_to_multiple_of, return_tensors, return_token_type_ids, return_attention_mask, return_overflowing_tokens, return_special_tokens_mask, return_offsets_mapping, return_length, verbose, **kwargs)
   2450         if not _is_valid_text_input(text):
   2451             raise ValueError(
-> 2452                 "text input must of type str (single example), List[str] (batch or single pretokenized example) "
   2453                 "or List[List[str]] (batch of pretokenized examples)."
   2454             )

ValueError: text input must of type str (single example), List[str] (batch or single pretokenized example) or List[List[str]] (batch of pretokenized examples).
```
Debugging this problem was not easy, all transcriptions in the dataset are correct strings. Finally I discovered that the transcription string 'null' is interpreted as [None] by the `load_dataset()` script. By deleting this row in the dataset the  training worked fine.

## Expected result:
transcription 'null' interpreted as 'str' instead of 'None'.

## Reproduction
Here is the code to reproduce the error with a one-row-dataset.

```
with open("null-test.csv") as f:
    reader = csv.reader(f)
    for row in reader:
        print(row)
```

['wav_filename', 'wav_filesize', 'transcript']
['wavs/female/NULL1.wav', '17530', 'null']

```
dataset = load_dataset('csv', data_files={'train': 'null-test.csv'})
```

Using custom data configuration default-81ac0c0e27af3514
Downloading and preparing dataset csv/default to /root/.cache/huggingface/datasets/csv/default-81ac0c0e27af3514/0.0.0/433e0ccc46f9880962cc2b12065189766fbb2bee57a221866138fb9203c83519...
Downloading data files: 100%
1/1 [00:00<00:00, 29.55it/s]
Extracting data files: 100%
1/1 [00:00<00:00, 23.66it/s]
Dataset csv downloaded and prepared to /root/.cache/huggingface/datasets/csv/default-81ac0c0e27af3514/0.0.0/433e0ccc46f9880962cc2b12065189766fbb2bee57a221866138fb9203c83519. Subsequent calls will reuse this data.
100%
1/1 [00:00<00:00, 25.84it/s]

```
print(dataset['train']['transcript'])
```

[None]

## Environment info
```
!pip install datasets==2.2.2
!pip install transformers==4.19.2
```


Optimize contiguous shard and select

I thought of just mentioning the benefits I got. Here's the code that @lhoestq provided:

```py
import os
from datasets import load_dataset
from tqdm.auto import tqdm

ds = load_dataset("squad", split="train")
os.makedirs("tmp")

num_shards = 5
for index in tqdm(range(num_shards)):
    size = len(ds) // num_shards
    shard = Dataset(ds.data.slice(size * index, size), fingerprint=f"{ds._fingerprint}_{index}")
    shard.to_json(f"tmp/data_{index}.jsonl")
```

It is 1.64s. Previously the code was:

```py
num_shards = 5
for index in tqdm(range(num_shards)):
    shard = ds.shard(num_shards=num_shards, index=index, contiguous=True)
    shard.to_json(f"tmp/data_{index}.jsonl")
    # upload_to_gcs(f"tmp/data_{index}.jsonl")
```

It was 2min31s.

I ran it on my humble MacBook Pro:

<img width="574" alt="image" src="https://user-images.githubusercontent.com/22957388/172864881-f1db489a-2305-47f2-a07f-7d3df610b1b8.png">

I addressed your comments @albertvillanova , let me know what you think :)

Currently `.shard()` and `.select()` always create an indices mapping. However if the requested data are contiguous, it's much more optimized to simply slice the Arrow table instead of building an indices mapping. In particular:
- the shard/select operation will be much faster
- reading speed will be much faster in the resulting dataset, since it won't have to do a lookup step in the indices mapping

Since `.shard()` is also used for `.map()` with `num_proc>1`, it will also significantly improve the reading speed of multiprocessed `.map()` operations

Here is an example of speed-up:
```python
>>> import io
>>> import numpy as np
>>> from datasets import Dataset
>>> ds = Dataset.from_dict({"a": np.random.rand(10_000_000)})
>>> shard = ds.shard(num_shards=4, index=0, contiguous=True)  # this calls `.select(range(2_500_000))`
>>> buf = io.BytesIO()
>>> %time dd.to_json(buf)
Creating json from Arrow format: 100%|██████████████████| 100/100 [00:00<00:00, 376.17ba/s]
CPU times: user 258 ms, sys: 9.06 ms, total: 267 ms
Wall time: 266 ms
```
while previously it was
```python
Creating json from Arrow format: 100%|███████████████████| 100/100 [00:03<00:00, 29.41ba/s]
CPU times: user 3.33 s, sys: 69.1 ms, total: 3.39 s
Wall time: 3.4 s
```

In this simple case the speed-up is x10, but @sayakpaul experienced a x100 speed-up on its data when exporting to JSON.

## Implementation details

I mostly improved `.select()`: it now checks if the input corresponds to a contiguous chunk of data and then it slices the main Arrow table (or the indices mapping table if it exists). To check if the input indices are contiguous it checks two possibilities:
- if the indices is of type `range`, it checks that start >= 0 and step = 1
- otherwise in the general case, it iterates over the indices. If all the indices are contiguous then we're good, otherwise we have to build an indices mapping.

Having to iterate over the indices doesn't cause performance issues IMO because:
- either they are contiguous and in this case the cost of iterating over the indices is much less than the cost of creating an indices mapping
- or they are not contiguous, and then iterating generally stops quickly when it first encounters the first indice that is not contiguous.


Fix bigbench config names

Fix https://github.com/huggingface/datasets/issues/4462 in the case of bigbench


Extend support for streaming datasets that use xml.dom.minidom.parse

This PR extends the support in streaming mode for datasets that use `xml.dom.minidom.parse`, by patching that function.

This PR adds support for streaming datasets like "Yaxin/SemEval2015".

Fix #4453.


Use config_id to check split sizes instead of config name

closing in favor of https://github.com/huggingface/datasets/pull/4465

Fix https://github.com/huggingface/datasets/issues/4462


BigBench: NonMatchingSplitsSizesError when passing a dataset configuration parameter

Something isn't working

Why not adding `max_examples` as part of the config name?

Yup it can also work, and maybe it's simpler this way. Opening a PR to fix bigbench instead of https://github.com/huggingface/datasets/pull/4463

Hi @lhoestq,

Thank you for taking a look at this issue, and proposing a solution.
Unfortunately, after trying the fix in #4465 I still see the same issue.

I think there is some subtlety where the config name gets overwritten somewhere when `BUILDER_CONFIGS`[(link)](https://github.com/huggingface/datasets/blob/master/datasets/bigbench/bigbench.py#L126) is defined.

If I print out the `self.config.name` in the current version (with the fix in #4465), I see just the task name, but if I comment out `BUILDER_CONFIGS`, the `num_shots` and `max_examples` gets appended as was meant by #4465.

I haven't managed to track down where this happens, but I thought you might know?

(Another comment on your fix: the `name` variable is used to fetch the task from the bigbench API, so modifying it causes an error if it's actually called. This can easily be fixed by having `config_name` variable in addition to the `task_name`)

As noticed in https://github.com/huggingface/datasets/pull/4125 when a dataset config class has a parameter that reduces the number of examples (e.g. named `max_examples`), then loading the dataset and passing `max_examples` raises `NonMatchingSplitsSizesError`.

This is because it will check for expected the number of examples of the config with the same name without taking into account the `max_examples` parameter. This can be fixed by checking the expected number of examples using the **config id** instead of name. Indeed the config id corresponds to the config name + an optional suffix that depends on the config parameters


AttributeError: module 'datasets' has no attribute 'load_dataset'

Something isn't working

## Describe the bug
I have piped install datasets, but this package doesn't have these attributes: load_dataset, load_metric.

## Environment info
- `datasets` version: 1.9.0
- Platform: Linux-5.13.0-44-generic-x86_64-with-debian-bullseye-sid
- Python version: 3.6.13
- PyArrow version: 6.0.1


Drop Python 3.6 support

I've disabled the `test_dummy_dataset_serialize_s3` tests in the Linux CI to avoid the failures (these tests only fail on Windows in 3.6). These failures are unrelated to this PR's changes, and I would like to address this in a new PR.

[This comment](https://github.com/pytorch/audio/issues/2363#issuecomment-1179089175) explains the issue with MP3 decoding in `torchaudio` in the latest release (supports Python 3.7+). I fixed CI by pinning `torchaudio` to `<0.12.0`. Another way to fix this issue is by installing `ffmpeg` with conda or using the unofficial GH action. But I don't think it's worth making CI more complex, considering we can wait for the soundfile release, which should bring MP3 decoding, and drop the `torchaudio` dependency then.

I think we can merge in this state. Also, if an env has Python version < 3.7 installed, we raise a warning, so I don't think we even need to create (and pin) an issue to notify the contributors of this change.

Remove the fallback imports/checks in the code needed for Python 3.6 and update the requirements/CI files. Also, use Python types for the NumPy dtype wherever possible to avoid deprecation warnings in newer NumPy versions.


Add and fix language tags for udhr dataset

Related to #4362.


First draft of the docs for TF + Datasets

This is probably quite close to being ready, so cc some TF people @gante @amyeroberts @merveenoyan just so they see it! No need for a full review, but if you have any comments or suggestions feel free.

Thanks ! We plan to make a new release later today for `to_tf_dataset` FYI, so I think we can merge it soon and include this documentation in the new release

I might cc a few of the other TF people to take a look when this is closer to being finished, but it's still a draft for now.


Workflow for Tabular data

enhancement

New feature or request

generic discussion

Generic discussion on the library

I use below to load a dataset:
```
dataset = datasets.load_dataset("scikit-learn/auto-mpg")
df = pd.DataFrame(dataset["train"])
```
TBH as said, tabular folk split their own dataset, they sometimes have two splits, sometimes three. Maybe somehow avoiding it for tabular datasets might be good for later. (it's just UX improvement)

Tabular data are treated very differently than data for NLP, audio, vision, etc. and therefore the worflow for tabular data in `datasets` is not ideal.

For example for tabular data, it is common to use pandas/spark/dask to process the data, and then load the data into X and y (X is an array of features and y an array of labels), then train_test_split and finally feed the data to a machine learning model.

In `datasets` the workflow is different: we use load_dataset, then map, then train_test_split (if we only have a train split) and we end up with columnar dataset splits, not formatted as X and y.

Right now, it is already possible to convert a dataset from and to pandas, but there are still many things that could improve the workflow for tabular data:
- be able to load the data into X and y
- be able to load a dataset from the output of spark or dask (as far as I know it's usually csv or parquet files on S3/GCS/HDFS etc.)
- support "unsplit" datasets explicitly, instead of putting everything in "train" by default

cc @adrinjalali @merveenoyan feel free to complete/correct this :)

Feel free to also share ideas of APIs that would be super intuitive in your opinion !


Update data URLs in fever dataset

_The documentation is not available anymore as the PR was closed or merged._

As stated in their website, data owners updated their URLs on 28/04/2022.

This PR updates the data URLs.

Fix #4452.


Dataset Viewer issue for Yaxin/SemEval2015

duplicate

This issue or pull request already exists

dataset-viewer

Related to the dataset viewer on huggingface.co

Closing since it's a duplicate of https://github.com/huggingface/datasets/issues/4453

### Link

_No response_

### Description

the link could not visit

### Owner

_No response_


I understand that the issue is that a remote file (URL) is being loaded as a local file. Right @albertvillanova @lhoestq?

```
Message:       [Errno 2] No such file or directory: 'https://raw.githubusercontent.com/YaxinCui/ABSADataset/main/SemEval2015Task12Corrected/train/restaurants_train.xml'
```

`xml.dom.minidom.parse` is not supported in streaming mode. I opened a PR here to fix it:
https://huggingface.co/datasets/Yaxin/SemEval2015/discussions/1

Please review the PR @WithYouTo and let me know if it works !

Additionally, I'm also patching our library, so that we support streaming datasets that use `xml.dom.minidom.parse`.

### Link

_No response_

### Description

_No response_

### Owner

_No response_


Trying to load FEVER dataset results in NonMatchingChecksumError

bug

Something isn't working

Thanks for reporting @santhnm2. We are fixing it.

Data owners updated their URLs recently. We have to align with them, otherwise you do not download anything (that is why ignore_verifications does not work).

## Describe the bug
Trying to load the `fever` dataset fails with `datasets.utils.info_utils.NonMatchingChecksumError`.

I tried with `download_mode="force_redownload"` but that did not fix the error. I also tried with `ignore_verification=True` but then that raised a `json.decoder.JSONDecodeError`.

## Steps to reproduce the bug
```python
from datasets import load_dataset

dataset = load_dataset('fever', 'v1.0') # Fails with NonMatchingChecksumError
dataset = load_dataset('fever', 'v1.0', download_mode="force_redownload") # Fails with NonMatchingChecksumError
dataset = load_dataset('fever', 'v1.0', ignore_verification=True)` # Fails with JSONDecodeError
```

## Expected results
I expect this call to return with no error raised.

## Actual results
With `ignore_verification=False`:
```
*** datasets.utils.info_utils.NonMatchingChecksumError: Checksums didn't match for dataset source files:
['https://s3-eu-west-1.amazonaws.com/fever.public/train.jsonl', 'https://s3-eu-west-1.amazonaws.com/fever.public/shared_task_dev.jsonl', 'https://s3-eu-west-1.amazonaws.com/fever.public/shared_task_dev_public.jsonl', 'https://s3-eu-west-1.amazonaws.com/fever.public/shared_task_test.jsonl', 'https://s3-eu-west-1.amazonaws.com/fever.public/paper_dev.jsonl', 'https://s3-eu-west-1.amazonaws.com/fever.public/paper_test.jsonl']
```
With `ignore_verification=True`:
```
*** json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version: 2.2.3.dev0
- Platform: Linux-4.15.0-50-generic-x86_64-with-glibc2.10
- Python version: 3.8.13
- PyArrow version: 8.0.0
- Pandas version: 1.4.2


Use newer version of multi-news with fixes

Closes #4430.


Update README.md of fquad


Rj

import android.content.DialogInterface;
import android.database.Cursor;
import android.os.Bundle;
import android.view.View;
import android.widget.ArrayAdapter;
import android.widget.Button;
import android.widget.EditText;
import android.widget.Toast;

import androidx.appcompat.app.AlertDialog;
import androidx.appcompat.app.AppCompatActivity;

public class MainActivity extends AppCompatActivity {

    private EditText editTextID;
    private EditText editTextName;
    private EditText editTextNum;

    private String name;
    private int number;
    private String ID;

    private dbHelper db;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        db = new dbHelper(this);

        editTextID = findViewById(R.id.editText1);
        editTextName = findViewById(R.id.editText2);
        editTextNum = findViewById(R.id.editText3);

        Button buttonSave = findViewById(R.id.button);
        Button buttonRead = findViewById(R.id.button2);
        Button buttonUpdate = findViewById(R.id.button3);
        Button buttonDelete = findViewById(R.id.button4);
        Button buttonSearch = findViewById(R.id.button5);
        Button buttonDeleteAll = findViewById(R.id.button6);

        buttonSave.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {

                name = editTextName.getText().toString();

                String num = editTextNum.getText().toString();

                if (name.isEmpty() || num.isEmpty()) {

                    Toast.makeText(MainActivity.this, "Cannot Submit Empty Fields", Toast.LENGTH_SHORT).show();
                } else {
                    number = Integer.parseInt(num);

                    try {
                        // Insert Data
                        db.insertData(name, number);

                        // Clear the fields
                        editTextID.getText().clear();
                        editTextName.getText().clear();
                        editTextNum.getText().clear();

                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }

            }
        });

        buttonRead.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {

                final ArrayAdapter<String> adapter = new ArrayAdapter<>(MainActivity.this, android.R.layout.simple_list_item_1);
                String name;
                String num;
                String id;

                try {

                    Cursor cursor = db.readData();
                    if (cursor != null && cursor.getCount() > 0) {

                        while (cursor.moveToNext()) {

                            id = cursor.getString(0); // get data in column index 0
                            name = cursor.getString(1); // get data in column index 1
                            num = cursor.getString(2); // get data in column index 2

                            // Add SQLite data to listView
                            adapter.add("ID :- " + id + "\n" +
                                    "Name :- " + name + "\n" +
                                    "Number :- " + num + "\n\n");

                        }

                    } else {

                        adapter.add("No Data");
                    }
                    cursor.close();

                } catch (Exception e) {
                    e.printStackTrace();
                }

                // show the saved data in alertDialog
                AlertDialog.Builder builder = new AlertDialog.Builder(MainActivity.this);
                builder.setTitle("SQLite saved data");
                builder.setIcon(R.mipmap.app_icon_foreground);
                builder.setAdapter(adapter, new DialogInterface.OnClickListener() {
                    @Override
                    public void onClick(DialogInterface dialog, int which) {

                    }
                });

                builder.setPositiveButton("OK", new DialogInterface.OnClickListener() {
                    @Override
                    public void onClick(DialogInterface dialog, int which) {

                        dialog.cancel();
                    }
                });

                AlertDialog dialog = builder.create();
                dialog.show();

            }
        });

        buttonUpdate.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {

                name = editTextName.getText().toString();

                String num = editTextNum.getText().toString();
                ID = editTextID.getText().toString();

                if (name.isEmpty() || num.isEmpty() || ID.isEmpty()) {

                    Toast.makeText(MainActivity.this, "Cannot Submit Empty Fields", Toast.LENGTH_SHORT).show();
                } else {
                    number = Integer.parseInt(num);

                    try {
                        // Update Data
                        db.updateData(ID, name, number);

                        // Clear the fields
                        editTextID.getText().clear();
                        editTextName.getText().clear();
                        editTextNum.getText().clear();

                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }

            }
        });

        buttonDelete.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {

                ID = editTextID.getText().toString();

                if (ID.isEmpty()) {

                    Toast.makeText(MainActivity.this, "Please enter the ID", Toast.LENGTH_SHORT).show();
                } else {

                    try {
                        // Delete Data
                        db.deleteData(ID);

                        // Clear the fields
                        editTextID.getText().clear();
                        editTextName.getText().clear();
                        editTextNum.getText().clear();

                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }

            }
        });

        buttonDeleteAll.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {

                // Delete all data
                // You can simply delete all the data by calling this method --> db.deleteAllData();
                // You can try this also
                AlertDialog.Builder builder = new AlertDialog.Builder(MainActivity.this);
                builder.setIcon(R.mipmap.app_icon_foreground);
                builder.setTitle("Delete All Data");
                builder.setCancelable(false);
                builder.setMessage("Do you really need to delete your all data ?");
                builder.setPositiveButton("Yes", new DialogInterface.OnClickListener() {
                    @Override
                    public void onClick(DialogInterface dialog, int which) {

                        // User confirmed , now you can delete the data
                        db.deleteAllData();

                        // Clear the fields
                        editTextID.getText().clear();
                        editTextName.getText().clear();
                        editTextNum.getText().clear();
                    }
                });
                builder.setNegativeButton("No", new DialogInterface.OnClickListener() {
                    @Override
                    public void onClick(DialogInterface dialog, int which) {

                        // user not confirmed
                        dialog.cancel();
                    }
                });

                AlertDialog dialog = builder.create();
                dialog.show();

            }
        });

        buttonSearch.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {

                ID = editTextID.getText().toString();

                if (ID.isEmpty()) {

                    Toast.makeText(MainActivity.this, "Please enter the ID", Toast.LENGTH_SHORT).show();
                } else {

                    try {
                        // Search data
                        Cursor cursor = db.searchData(ID);
                        if (cursor.moveToFirst()) {

                            editTextName.setText(cursor.getString(1));
                            editTextNum.setText(cursor.getString(2));
                            Toast.makeText(MainActivity.this, "Data successfully searched", Toast.LENGTH_SHORT).show();

                        } else {
                            Toast.makeText(MainActivity.this, "ID not found", Toast.LENGTH_SHORT).show();

                            editTextNum.setText("ID Not found");
                            editTextName.setText("ID not found");

                        }

                        cursor.close();

                    } catch (Exception e) {
                        e.printStackTrace();

                    }
                }

            }
        });
    }
}


New Preprocessing Feature  - Deduplication [Request]

enhancement

New feature or request

Hi! The [datasets_sql](https://github.com/mariosasko/datasets_sql) package lets you easily find distinct rows in a dataset (an example with `SELECT DISTINCT` is in the readme). Deduplication is (still) not part of the official API because it's hard to implement for datasets bigger than RAM while only using the native PyArrow ops.

(Btw, this is a duplicate of https://github.com/huggingface/datasets/issues/2514)

**Is your feature request related to a problem? Please describe.**
Many large datasets are full of duplications and it has been shown that deduplicating datasets can lead to better performance while training, and more truthful evaluation at test-time.

A feature that allows one to easily deduplicate a dataset can be cool!

**Describe the solution you'd like**
We can define a function and keep only the first/last data-point that yields the value according to this function.

**Describe alternatives you've considered**
The clear alternative is to repeat a clear boilerplate every time someone want to deduplicate a dataset.


Minor fixes/improvements in `scene_parse_150` card

Add `paperswithcode_id` and fix some links in the `scene_parse_150` card.


Add missing kwargs to docstrings


Fix missing args in docstring of load_dataset_builder

Currently, the docstring of `load_dataset_builder` only contains the first parameter `path` (no other):
- https://huggingface.co/docs/datasets/v2.2.1/en/package_reference/loading_methods#datasets.load_dataset_builder.path


Fix kwargs in docstrings

To fix the rendering of `**kwargs` in docstrings, a parentheses must be added afterwards.

See:
- huggingface/doc-builder/issues/235


Dataset Viewer issue for openclimatefix/nimrod-uk-1km

If I understand correctly, this is due to the key `split` missing in the line https://huggingface.co/datasets/openclimatefix/nimrod-uk-1km/blob/main/nimrod-uk-1km.py#L41 of the script.
Maybe @albertvillanova could confirm.

Indeed there are several issues in this dataset loading script.

The one pointed out by @severo: for the default configuration "crops": https://huggingface.co/datasets/openclimatefix/nimrod-uk-1km/blob/main/nimrod-uk-1km.py#L244
- The download manager downloads `_URL`
- But `_URL` is not defined:  https://huggingface.co/datasets/openclimatefix/nimrod-uk-1km/blob/main/nimrod-uk-1km.py#L41
  ```python
  _URL = {'train': []}
  ```
- Afterwards, for each split, a different key in `_ULR` is used, but it only contains one key: "train"
  - "valid" key: https://huggingface.co/datasets/openclimatefix/nimrod-uk-1km/blob/main/nimrod-uk-1km.py#L260
  - "test key: https://huggingface.co/datasets/openclimatefix/nimrod-uk-1km/blob/main/nimrod-uk-1km.py#L269

These keys do not exist inside `_URL`, thus the error message reported in the viewer:
```
Exception:     KeyError
Message:       'valid'
```

Would anyone want to submit a Hub PR (or open a Discussion for the authors to be aware) to this dataset? https://huggingface.co/datasets/openclimatefix/nimrod-uk-1km

Hi, I'm the main author for that dataset, so I'll work on updating it! I was working on debugging some stuff awhile ago, which is what broke it.

I've opened a Discussion page, so that we can ask/answer and propose fixes until the script works properly: https://huggingface.co/datasets/openclimatefix/nimrod-uk-1km/discussions/1

CC: @julien-c @jacobbieker


Dataset Viewer issue for amazon_polarity

Not sure what happened 😬, but it's fixed

### Link

https://huggingface.co/datasets/amazon_polarity/viewer/amazon_polarity/test

### Description

For some reason the train split is OK but the test split is not for this dataset:

```
Server error
Status code:   400
Exception:     FileNotFoundError
Message:       [Errno 2] No such file or directory: '/cache/modules/datasets_modules/datasets/amazon_polarity/__init__.py'
```

### Owner

No


Dataset Viewer issue for aeslc

### Link

https://huggingface.co/datasets/aeslc

### Description

The dataset viewer can't find `dataset_infos.json` in it's cache:

```
Server error
Status code:   400
Exception:     FileNotFoundError
Message:       [Errno 2] No such file or directory: '/cache/modules/datasets_modules/datasets/aeslc/eb8e30234cf984a58ebe9f205674597ac1db2ec91e7321cd7f36864f7e3671b8/dataset_infos.json'
```

### Owner

No


Update docs around audio and vision

> Let me know what you think, especially if we should include some code samples for training a model in the audio/vision sections. I left this out since we already showed it in the NLP section. I want to keep the focus on using Datasets to load and process a dataset, and not so much the training part. Maybe we can add links to the Transformers docs instead?

We plan to address this with end-to-end examples (for each modality) more focused on preprocessing than the ones in the Transformers docs.

As part of the strategy to center the docs around the different modalities, this PR updates the quickstart to include audio and vision examples. This improves the developer experience by making audio and vision content more discoverable, enabling users working in these modalities to also quickly get started without digging too deeply into the docs.

Other changes include:

- Moved the installation guide to the Get Started section because it should be part of a user's onboarding to the library before exploring tutorials or how-to's.
- Updated the native TF code at creating a `tf.data.Dataset` because it was throwing an error. The `to_tensor()` bit was redundant and removing it fixed the error (please double-check me here!).
- Added some UI components to the quickstart so it's easier for users to navigate directly to the relevant section with context about what to expect.
- Reverted to the code tabs for content that don't have any framework-specific text. I think this saves space compared to the code blocks. We'll still use the code blocks if the `torch` text is different from the `tf` text.

Let me know what you think, especially if we should include some code samples for training a model in the audio/vision sections. I left this out since we already showed it in the NLP section. I want to keep the focus on using Datasets to load and process a dataset, and not so much the training part. Maybe we can add links to the Transformers docs instead?


TIMIT won't load after manual download: Errors about files that don't exist

bug

Something isn't working

To have some context, please see:
- #4145

Please, also note that we have recently made some fixes to the script, which are in our GitHub master branch but not yet released:
- #4422
- #4425
- #4436

Thanks Albert! I'll try pulling `datasets` from the git repo instead of PyPI, and/or just wait for the next release.

I'm closing this issue then. Please, feel free to reopen it again if the problem persists.

## Describe the bug

I get the message from HuggingFace that it must be downloaded manually. From the URL provided in the message, I got to UPenn page for manual download. (UPenn apparently want $250? for the dataset??)   ...So, ok, I obtained a copy from a friend and also a smaller version from Kaggle. But in both cases the HF dataloader fails; it is looking for files that don't exist anywhere in the dataset: it is looking for files with lower-case letters like "**test*" (all the filenames in both my copies are uppercase) and certain file extensions that exclude the .DOC which is provided in TIMIT:

## Steps to reproduce the bug
```python
data = load_dataset('timit_asr', 'clean')['train']
```

## Expected results
The dataset should load with no errors.

## Actual results
This error message:
```
  File "/home/ubuntu/envs/data2vec/lib/python3.9/site-packages/datasets/data_files.py", line 201, in resolve_patterns_locally_or_by_urls
    raise FileNotFoundError(error_msg)
FileNotFoundError: Unable to resolve any data file that matches '['**test*', '**eval*']' at /home/ubuntu/datasets/timit with any supported extension ['csv', 'tsv', 'json', 'jsonl', 'parquet', 'txt', 'blp', 'bmp', 'dib', 'bufr', 'cur', 'pcx', 'dcx', 'dds', 'ps', 'eps', 'fit', 'fits', 'fli', 'flc', 'ftc', 'ftu', 'gbr', 'gif', 'grib', 'h5', 'hdf', 'png', 'apng', 'jp2', 'j2k', 'jpc', 'jpf', 'jpx', 'j2c', 'icns', 'ico', 'im', 'iim', 'tif', 'tiff', 'jfif', 'jpe', 'jpg', 'jpeg', 'mpg', 'mpeg', 'msp', 'pcd', 'pxr', 'pbm', 'pgm', 'ppm', 'pnm', 'psd', 'bw', 'rgb', 'rgba', 'sgi', 'ras', 'tga', 'icb', 'vda', 'vst', 'webp', 'wmf', 'emf', 'xbm', 'xpm', 'zip']
```

But this is a strange sort of error: why is it looking for lower-case file names when all the TIMIT dataset filenames are uppercase?  Why does it exclude .DOC files when the only parts of the TIMIT data set with "TEST" in them have ".DOC" extensions?   ...I wonder, how was anyone able to get this to work in the first place?

The files in the dataset look like the following:
```
³       PHONCODE.DOC
³       PROMPTS.TXT
³       SPKRINFO.TXT
³       SPKRSENT.TXT
³       TESTSET.DOC
```
...so why are these being excluded by the dataset loader?

## Environment info
- `datasets` version: 2.2.2
- Platform: Linux-5.4.0-1060-aws-x86_64-with-glibc2.27
- Python version: 3.9.9
- PyArrow version: 8.0.0
- Pandas version: 1.4.2


Fix docstring of inspect_dataset

_The documentation is not available anymore as the PR was closed or merged._

As pointed out by @sgugger:
- huggingface/doc-builder/issues/235


Add missing columns to `blended_skill_talk`

Adds the missing columns to `blended_skill_talk` to align the loading logic with [ParlAI](https://github.com/facebookresearch/ParlAI/blob/main/parlai/tasks/blended_skill_talk/build.py).

Fix #4426


Fix directory names for LDC data in timit_asr dataset

Related to:
- #4422


Load a local cached dataset that has been modified

Something isn't working

Hi! `datasets` caches every modification/loading, so you can either rerun the pipeline up to the `map` call or use `Dataset.from_file(modified_dataset)` to load the dataset directly from the cache file.

## Describe the bug
I have loaded a dataset as follows:
```
d = load_dataset("emotion", split="validation")
```
Afterwards I make some modifications to the dataset via a `map` call:
```
d.map(some_update_func, cache_file_name=modified_dataset)
```
This generates a cached version of the dataset on my local system in the same directory as the original download of the data (/path/to/cache). Running an `ls` returns:
```
modified_dataset
dataset_info.json
emotion-test.arrow
emotion-train.arrow
emotion-validation.arrow
```
as expected. However, when I try to load up the modified cached dataset via a call to
```
modified = load_dataset("emotion", split="validation", data_files="/path/to/cache/modified_dataset")
```
it simply redownloads a new version of the dataset and dumps to a new cache rather than loading up the original modified dataset:
```
Using custom data configuration validation-cdbf51685638421b
Downloading and preparing dataset emotion/validation to ...
```

How am I supposed to load the original modified local cache copy of the dataset?

## Environment info
- `datasets` version: 2.2.2
- Platform: Linux-5.4.0-113-generic-x86_64-with-glibc2.17
- Python version: 3.8.13
- PyArrow version: 8.0.0
- Pandas version: 1.4.2


Fix dummy dataset generation script for handling nested types of _URLs

It seems that when user specify nested _URLs structures in their dataset script. An error will be raised when generating dummy dataset.

I think the types of all elements in `dummy_data_dict.values()` should be checked because they may have different types.

Linked to issue #4428

PS: I am not sure whether my code fix this issue in a proper way.


Fix script fetching and local path handling in `inspect_dataset` and `inspect_metric`

Added back the `[:]` and a comment to explain why this is needed.

Fix #4348


Fix builder docstring

Currently, the args of `DatasetBuilder` do not appear in the docs: https://huggingface.co/docs/datasets/v2.1.0/en/package_reference/builder_classes#datasets.DatasetBuilder


Add personaldialog datasets

These test errors are related to issue #4428

I only made a trivial modification in my commit https://github.com/huggingface/datasets/pull/4431/commits/402c893d35224d7828176717233909ac5f1e7b3e

I have submitted a PR #4434 for the about issue.

> Awesome thanks for adding this dataset :)
>
> I just have one comment about the licensing.
>
> Also it seems that you already have the dataset in https://huggingface.co/datasets/silver/personal_dialog, so it's unnecessary to add it here

Thank you very much for your comment.

So, should I close this PR?

Thanks for fixing the licensing section :)

> So, should I close this PR?

Yes you can close this PR, it's better if your dataset is under your namespace at https://huggingface.co/datasets/silver/personal_dialog :)

Don't forget to update the licensing section on https://huggingface.co/datasets/silver/personal_dialog as well

It seems that all tests are passed


Add ability to load newer, cleaner version of Multi-News

enhancement

New feature or request

Hi! Our versioning is based on Git revisions (the `revision` param in `load_dataset`), so you can just replace the old URL with the new one and open a PR :). I can also give you some pointers if needed.

@mariosasko Awesome thanks! I will do that. Looks like this new version of the data is not available as a zip but as three files (train/dev/test). How is this usually handled in HF Datasets, should `_URL` be a dict with keys `train`, `val`, `test` perhaps?

Yes! Let me help you with more detailed instructions.

In the first step, we need to update the URLs. One of the possible dictionary structures is as follows:
```python
_URLs = {
    "train": {"src": "https://drive.google.com/uc?export=download&id=1wHAWDOwOoQWSj7HYpyJ3Aeud8WhhaJ7P", "tgt": "https://drive.google.com/uc?export=download&id=1QVgswwhVTkd3VLCzajK6eVkcrSWEK6kq"}
    "val": ...
    "test": ...
}
```

(You can use this page to generate direct download links: https://sites.google.com/site/gdocs2direct/)

Then we move to the `split_generators` method:
```python
def _split_generators(self, dl_manager):
    """Returns SplitGenerators."""
    files = dl_manager.download(_URLs)
    return [
        datasets.SplitGenerator(
            name=datasets.Split.TRAIN,
            gen_kwargs={"src_file": files["train"]["src"], "tgt_file": files["train"]["tgt"]},
        ),
        ... # same for val and test
    ]
```
Finally, we adjust the signature of `_generate_examples`:
```python
def _generate_examples(self, src_file, tgt_file):
     """Yields examples."""
     with open(src_file, encoding="utf-8") as src_f, open(
         tgt_file, encoding="utf-8"
     ) as tgt_f:
         ... # the rest is the same
```

And that's it!

PS: Let me know if you need help updating the dummy data and regenerating the metadata file.

Awesome! Thanks for the detailed help, that was straightforward with your instruction. However, I think I am being blocked by this issue: https://github.com/huggingface/datasets/issues/4428

Feel free to open a PR, and I can fix this manually.

**Is your feature request related to a problem? Please describe.**

The [Multi-News dataloader points to the original version of the Multi-News dataset](https://github.com/huggingface/datasets/blob/12540dd75015678ec6019f258d811ee107439a73/datasets/multi_news/multi_news.py#L47), but this has [known errors in it](https://github.com/Alex-Fabbri/Multi-News/issues/11). There exists a [newer version which fixes some of these issues](https://drive.google.com/open?id=1jwBzXBVv8sfnFrlzPnSUBHEEAbpIUnFq).

Unfortunately I don't think you can just replace this old URL with the new one, otherwise this could lead to issues with reproducibility.

**Describe the solution you'd like**

Add a new version to the Multi-News dataloader that points to the updated dataset which has fixes for some known issues.

**Describe alternatives you've considered**

Replace the current URL to the original version to the dataset with the URL to the version with fixes.

**Additional context**

Would be happy to make a PR for this, could someone maybe point me to another dataloader that has multiple versions so I can see how this is handled in `datasets`?


Update builder docstring for deprecated/added arguments

@mishig25 is investigating why deprecated/added do not affect the enclosed text format when used in args docstring: no special formatting appears:
- https://moon-ci-docs.huggingface.co/docs/datasets/pr_4429/en/package_reference/builder_classes#datasets.DatasetBuilder

@albertvillanova please check now 👍
https://moon-ci-docs.huggingface.co/docs/datasets/pr_4429/en/package_reference/builder_classes#datasets.DatasetBuilder

<img width="500" alt="Screenshot 2022-06-06 at 10 20 34" src="https://user-images.githubusercontent.com/11827707/172123471-fab97138-c903-4a71-ab7f-c90e5e43c58f.png">

Thanks @mishig25.

Just one question: is it expected to have the deprecated box right edge not filling all the page width (contrary to the added box)?

> Just one question: is it expected to have the deprecated box right edge not filling all the page width (contrary to the added box)?

Yes, that is expected 😊 because the depreacted box is being bounded by its parent box (the box for `name` argument in the screenshot above)

This PR updates the builder docstring with deprecated/added directives for arguments name/config_name.

Follow up of:
- #4414
- huggingface/doc-builder#233

First merge:
- #4432


Errors when building dummy data if you use nested _URLS

bug

Something isn't working

## Describe the bug
When making dummy data with the `datasets-cli dummy_data` tool,
an error will be raised if you use a nested _URLS in your dataset script.

Traceback (most recent call last):
  File "/home/name/LCCC/datasets/src/datasets/commands/datasets_cli.py", line 43, in <module>
    main()
  File "/home/name/LCCC/datasets/src/datasets/commands/datasets_cli.py", line 39, in main
    service.run()
  File "/home/name/LCCC/datasets/src/datasets/commands/dummy_data.py", line 311, in run
    self._autogenerate_dummy_data(
  File "/home/name/LCCC/datasets/src/datasets/commands/dummy_data.py", line 337, in _autogenerate_dummy_data
    dataset_builder._split_generators(dl_manager)
  File "/home/name/.cache/huggingface/modules/datasets_modules/datasets/personal_dialog/559332bced5eeafa7f7efc2a7c10ce02cee2a8116bbab4611c35a50ba2715b77/personal_dialog.py", line 108, in _split_generators
    data_dir = dl_manager.download_and_extract(urls)
  File "/home/name/LCCC/datasets/src/datasets/commands/dummy_data.py", line 56, in download_and_extract
    dummy_output = self.mock_download_manager.download(url_or_urls)
  File "/home/name/LCCC/datasets/src/datasets/download/mock_download_manager.py", line 130, in download
    return self.download_and_extract(data_url)
  File "/home/name/LCCC/datasets/src/datasets/download/mock_download_manager.py", line 122, in download_and_extract
    return self.create_dummy_data_dict(dummy_file, data_url)
  File "/home/name/LCCC/datasets/src/datasets/download/mock_download_manager.py", line 165, in create_dummy_data_dict
    if isinstance(first_value, str) and len(set(dummy_data_dict.values())) < len(dummy_data_dict.values()):
TypeError: unhashable type: 'list'

## Steps to reproduce the bug

You can use my dataset script implemented here:
https://github.com/silverriver/datasets/blob/2ecd36760c40b8e29b1137cd19b5bad0e19c76fd/datasets/personal_dialog/personal_dialog.py

```python
datasets_cli dummy_data datasets/personal_dialog --auto_generate
```

You can change https://github.com/silverriver/datasets/blob/2ecd36760c40b8e29b1137cd19b5bad0e19c76fd/datasets/personal_dialog/personal_dialog.py#L54
to

```
"train": "https://huggingface.co/datasets/silver/personal_dialog/resolve/main/dev_random.jsonl.gz"
```

before runing the above script to avoid downloading a large training data.

## Expected results
The dummy data should be generated

## Actual results
An error is raised.

It seems that in https://github.com/huggingface/datasets/blob/12540dd75015678ec6019f258d811ee107439a73/src/datasets/download/mock_download_manager.py#L165
We only check if the first item of dummy_data_dict.values() is str.
However, dummy_data_dict.values() may have the type of [str, list, list].
A simple fix would be changing https://github.com/huggingface/datasets/blob/12540dd75015678ec6019f258d811ee107439a73/src/datasets/download/mock_download_manager.py#L165 to

```python
if all([isinstance(value, str) for value in dummy_data_dict.values()]) and len(set(dummy_data_dict.values())) < len(dummy_data_dict.values()):
```

But I don't know if this kinds of change may bring any side effect since I am not sure about the detail logic here.

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- Platform: Linux
- Python version: Python 3.9.10
- PyArrow version: 7.0.0


Add HF.co for PRs/Issues for specific datasets

As in https://github.com/huggingface/transformers/pull/17485, issues and PR for datasets under a namespace have to be on the HF Hub


Add loading variable number of columns for different splits

enhancement

New feature or request

Hi! Indeed the column is missing, but you shouldn't get an error? Have you made some modifications (locally) to the loading script? I've opened a PR to add the missing columns to the script.

**Is your feature request related to a problem? Please describe.**
The original dataset `blended_skill_talk` consists of different sets of columns for the different splits: (test/valid) splits have additional data column `label_candidates` that the (train) doesn't have.
When loading such data, an exception occurs at table.py:cast_table_to_schema, because of mismatched columns.


Make extensions case-insensitive in timit_asr dataset

_The documentation is not available anymore as the PR was closed or merged._

Related to #4422.


Fix DuplicatedKeysError in timit_asr dataset

Fix #4422.


Add new dataset MMChat

Thanks ! As for https://github.com/huggingface/datasets/pull/4431 please also update the licensing section in https://huggingface.co/datasets/silver/mmchat ;)

Then if it's fine for you feel free to close this PR

Hi, I am adding a new dataset MMChat.
It seems that all tests are passed


Cannot load timit_asr data set

bug

Something isn't working

Thanks for reporting, @bhaddow.

I'm fixing it.

@bhaddow we have also made a fix so that you don't have to convert to uppercase the file extensions of the LDC data.

Would you mind checking if it works OK now for you and reporting if there are any issues? Thanks.

Hi @albertvillanova  -It loads fine on a copy of the data from deepai - although I have to remove the copies of the .WAV files (with extension .WAV,wav). On a copy of the data that was obtained from the LDC, the glob still fails to find the files. The LDC copy looks like it was copied from CD, in 2004, so the structure may be different to a current download.

Ah, if I change the train/ and test/ directories to TRAIN/ and TEST/  then it works!

Thanks for your investigation and report, @bhaddow. I'm adding another fix for the TRAIN/train and TEST/test directory names.

## Describe the bug
I am trying to load the timit_asr data set. I have tried with a copy from the LDC, and a copy from deepai. In both cases they fail with a "duplicate key" error. With the LDC version I have to convert the file extensions all to upper-case before I can load it at all.

## Steps to reproduce the bug
```python
timit = datasets.load_dataset("timit_asr", data_dir = "/path/to/dataset")
# Sample code to reproduce the bug
```

## Expected results

The data set should load without error. It worked for me before the LDC url change.

## Actual results
```
datasets.keyhash.DuplicatedKeysError: FAILURE TO GENERATE DATASET !
Found duplicate Key: SA1
Keys should be unique and deterministic in nature
```

## Environment info
<!-- You can run the command `datasets-cli env` and copy-and-paste its output below. -->
- `datasets` version:
- `datasets` version: 2.2.2
- Platform: Linux-5.4.0-90-generic-x86_64-with-glibc2.17
- Python version: 3.8.12
- PyArrow version: 8.0.0
- Pandas version: 1.4.2


Add extractor for bzip2-compressed files

This change enables loading bzipped datasets, just like any other compressed dataset.


Metric evaluation problems in multi-node, shared file system

Something isn't working

If you call `metric.compute` in a distributed setup like yours, then `metric.compute` is called in each process. `metric.compute` first calls `metric.add_batch`, and it looks like your error appears at that stage.

To make sure that all the processes have started writing their predictions/references at the same time, each process waits for process 0 to lock `slurm-{world_size}-0.arrow.lock`. Process 0 locks this file when `metric.add_batch` is called, so here when `metric.compute` is called.

Therefore your error can happen when process 0 takes too much time to call `metric.compute` compared to process 3 (>100 seconds by default). I haven't tried running your code but could it be the case ?

I guess it could also happen if you run multiple times the same distributed job at the same time with the same `experiment_id` because they would collide.

We've finally been able to isolate the problem, it wasn't a timing problem, but rather a file locking one.
The locks produced by calling `flock` where not visible between nodes (so the master node couldn't check other node's locks nor the other way around).

We are now having issues with the pre-processing in our runner script, but are not related with the rendezvous process during the evaluation phase. We will let you know about it once we address it.

Our solution to the rendezvous is as follows:
- We solved the problem by calling `lockf` instead of `flock`.
- We had to change slightly the `_check_all_processes_locks` method so that the main process (i.e. process 0) didn't check it's own lock (because `lockf` permits recursive locks and thus checking it only replaced the current lock with a new one).

We use a shared file system between nodes using GPFS in our cluster setup. Maybe the difference between the behavior we see with respect to your usage in multi-node executions comes from that fact. Which file system scheme do you use for the multi-node executions?

`lockf` seems to work in more settings than `flock`, so maybe we could write a PR so you could test it in your environment.

Cool, I'm glad you managed to make evaluation work :)

I'm not completely aware of the differences between lockf and flock, but I've read somewhere that flock is preferable over lockf in multithreading and multiprocessing situations. Here we definitely are in such a situation so unless it is super important I don't think we will switch to lockf

## Describe the bug
Metric evaluation fails in multi-node within a shared file system, because the master process cannot find the lock files from other nodes. (This issue was originally mentioned in the transformers repo https://github.com/huggingface/transformers/issues/17412)

## Steps to reproduce the bug

1. clone [this huggingface model](https://huggingface.co/PereLluis13/wav2vec2-xls-r-300m-ca-lm) and replace the `run_speech_recognition_ctc.py` script with the version in the gist [here](https://gist.github.com/gullabi/3f66094caa8db1c1e615dd35bd67ec71#file-run_speech_recognition_ctc-py).
2. Setup the `venv` according to the requirements of the model file plus `datasets==2.0.0`, `transformers==4.18.0` and `torch==1.9.0`
3. Launch the runner in a distributed environment which has a shared file system for two nodes, preferably with SLURM. Example [here](https://gist.github.com/gullabi/3f66094caa8db1c1e615dd35bd67ec71)

Specifically for the datasets, for the distributed setup the `load_metric` is called as:
```
    process_id=int(os.environ["RANK"])
    num_process=int(os.environ["WORLD_SIZE"])
    eval_metrics = {metric: load_metric(metric,
                                        process_id=process_id,
                                        num_process=num_process,
                                        experiment_id="slurm")
                    for metric in data_args.eval_metrics}
```

## Expected results
The training should not fail, due to the failure of the `Metric.compute()` step.

## Actual results
For the test I am executing the world size is 4, with 2 GPUs in 2 nodes. However the process is not finding the necessary lock files
```
  File "/gpfs/projects/bsc88/speech/asr/wav2vec2-xls-r-300m-ca-lm/run_speech_recognition_ctc.py", line 841, in <module>
    main()
  File "/gpfs/projects/bsc88/speech/asr/wav2vec2-xls-r-300m-ca-lm/run_speech_recognition_ctc.py", line 792, in main
    train_result = trainer.train(resume_from_checkpoint=checkpoint)
  File "/gpfs/projects/bsc88/projects/speech-tech-resources/venv_amd_speech/lib/python3.7/site-packages/transformers/trainer.py", line 1497, in train
    self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
  File "/gpfs/projects/bsc88/projects/speech-tech-resources/venv_amd_speech/lib/python3.7/site-packages/transformers/trainer.py", line 1624, in _maybe_log_save_evaluate
    metrics = self.evaluate(ignore_keys=ignore_keys_for_eval)
  File "/gpfs/projects/bsc88/projects/speech-tech-resources/venv_amd_speech/lib/python3.7/site-packages/transformers/trainer.py", line 2291, in evaluate
    metric_key_prefix=metric_key_prefix,
  File "/gpfs/projects/bsc88/projects/speech-tech-resources/venv_amd_speech/lib/python3.7/site-packages/transformers/trainer.py", line 2535, in evaluation_loop
    metrics = self.compute_metrics(EvalPrediction(predictions=all_preds, label_ids=all_labels))
  File "/gpfs/projects/bsc88/speech/asr/wav2vec2-xls-r-300m-ca-lm/run_speech_recognition_ctc.py", line 742, in compute_metrics
    metrics = {k: v.compute(predictions=pred_str, references=label_str) for k, v in eval_metrics.items()}
  File "/gpfs/projects/bsc88/speech/asr/wav2vec2-xls-r-300m-ca-lm/run_speech_recognition_ctc.py", line 742, in <dictcomp>
    metrics = {k: v.compute(predictions=pred_str, references=label_str) for k, v in eval_metrics.items()}
  File "/gpfs/projects/bsc88/projects/speech-tech-resources/venv_amd_speech/lib/python3.7/site-packages/datasets/metric.py", line 419, in compute
    self.add_batch(**inputs)
  File "/gpfs/projects/bsc88/projects/speech-tech-resources/venv_amd_speech/lib/python3.7/site-packages/datasets/metric.py", line 465, in add_batch
    self._init_writer()
  File "/gpfs/projects/bsc88/projects/speech-tech-resources/venv_amd_speech/lib/python3.7/site-packages/datasets/metric.py", line 552, in _init_writer
    self._check_rendez_vous()  # wait for master to be ready and to let everyone go
  File "/gpfs/projects/bsc88/projects/speech-tech-resources/venv_amd_speech/lib/python3.7/site-packages/datasets/metric.py", line 342, in _check_rendez_vous
    ) from None
ValueError: Expected to find locked file /home/bsc88/bsc88474/.cache/huggingface/metrics/wer/default/slurm-4-0.arrow.lock from process 3 but it doesn't exist.
```

When I look at the cache directory, I can see all the lock files in principle:
```
/home/bsc88/bsc88474/.cache/huggingface/metrics/wer/default/slurm-4-0.arrow
/home/bsc88/bsc88474/.cache/huggingface/metrics/wer/default/slurm-4-0.arrow.lock
/home/bsc88/bsc88474/.cache/huggingface/metrics/wer/default/slurm-4-1.arrow
/home/bsc88/bsc88474/.cache/huggingface/metrics/wer/default/slurm-4-1.arrow.lock
/home/bsc88/bsc88474/.cache/huggingface/metrics/wer/default/slurm-4-2.arrow
/home/bsc88/bsc88474/.cache/huggingface/metrics/wer/default/slurm-4-2.arrow.lock
/home/bsc88/bsc88474/.cache/huggingface/metrics/wer/default/slurm-4-3.arrow
/home/bsc88/bsc88474/.cache/huggingface/metrics/wer/default/slurm-4-3.arrow.lock
/home/bsc88/bsc88474/.cache/huggingface/metrics/wer/default/slurm-4-rdv.lock
```

I see that there was another related issue here https://github.com/huggingface/datasets/issues/1942, but it seems to have resolved via https://github.com/huggingface/datasets/pull/1966. Let me know if there is problem with how I am calling the `load_metric` or whether I need to make changes to the `.compute()` steps.

## Environment info
- `datasets` version: 2.0.0
- Platform: Linux-4.18.0-147.8.1.el8_1.x86_64-x86_64-with-centos-8.1.1911-Core
- Python version: 3.7.4
- PyArrow version: 7.0.0
- Pandas version: 1.3.0


Update `unittest` assertions over tuples from `assertEqual` to `assertTupleEqual`

enhancement

New feature or request

Hi! If the only goal is to improve readability, it's better to use `assertTupleEqual` than `assertSequenceEqual` for Python tuples. Also, note that this function is called internally by `assertEqual`, but I guess we can accept a PR to be more verbose.

Hi @mariosasko, right! I'll update the issue title/desc with `assertTupleEqual` even though as you said it seems to be internally using `assertEqual` so I'm not sure whether it's worth it or not...

https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertTupleEqual

I thought we were supposed to move gradually from `unittest` to `pytest`...

**Is your feature request related to a problem? Please describe.**

So this is more a readability improvement rather than a proposal, wouldn't it be better to use `assertTupleEqual` over the tuples rather than `assertEqual`? As `unittest` added that function in `v3.1`, as detailed at https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertTupleEqual, so maybe it's worth updating.

Find an example of an `assertEqual` over a tuple in 🤗 `datasets` unit tests over an `ArrowDataset` at https://github.com/huggingface/datasets/blob/0bb47271910c8a0b628dba157988372307fca1d2/tests/test_arrow_dataset.py#L570

**Describe the solution you'd like**

Start slowly replacing all the `assertEqual` statements with `assertTupleEqual` if the assertion is done over a Python tuple, as we're doing with the Python lists using `assertListEqual` rather than `assertEqual`.

**Additional context**

If so, please let me know and I'll try to go over the tests and create a PR if applicable, otherwise, if you consider this should stay as `assertEqual` rather than `assertSequenceEqual` feel free to close this issue! Thanks 🤗


Add dataset MMChat