[docs] Attention backends

[stevhliu]

Docs for enabling different attention backends.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[sayakpaul]

Thanks for starting this! Looking good. Before talking about specific attention backends, we could educate the users about the forms in which they can use them.

1. Through setting attention backend name model. set_attention_backend("<backend_name>").
2. Through the context manager.

Then we could maintain a table of attention backend names, their GitHub/official pages, and notes.

We could then take one example for a backend and make it complete.

This way, I think it will be leaner and easier to follow. WDYT?

[sayakpaul]

Suggested change

	The example below demonstrates how to enable the `_flash_3_hub` implementation. The [kernel](https://github.com/huggingface/kernels) library allows you to instantly use optimized compute kernels from the Hub without requiring any setup.
	The example below demonstrates how to enable the `_flash_3_hub` implementation for Flash Attention 3. The [kernels](https://github.com/huggingface/kernels) library allows you to instantly use optimized compute kernels from the Hub without requiring any setup.

But we should aso note the hardware restrictions in using FA3 as it's not supported in non-hopper architectures. In that case, regular Flash should be used through set_attention_backend("flash").

[sayakpaul]

1. Flex uses a different path in torch:
   

   diffusers/src/diffusers/models/attention_dispatch.py

   Line 847 in 5e181ed

   out = flex_attention.flex_attention(

2. The backends that leverage nn.functional.scaled_dot_product_attention() can be found in https://github.com/huggingface/diffusers/blob/5e181eddfe7e44c1444a2511b0d8e21d177850a0/src/diffusers/models/attention_dispatch.py (search with scaled_dot_product_attention().