HDDS-11508. Decouple delete batch limits from Ratis request size for DirectoryDeletingService.

[sadanand48]

What changes were proposed in this pull request?

Currently we limit the number of deletion objects (directories) in each run of DirectoryDeletingService based on Ratis request limit i.e If Ratis request proto size exceeds the limit , DDS stops sending further dirs for purging in the same run which means the task limit per run is not honoured. This PR fixes this by batching the requests as per Ratis proto limit and sending mutiple purge requests in the same run. This way the task limit per run is honoured.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-11508

How was this patch tested?

Unit tests

[errose28]

Thanks for working on this @sadanand48. Are you planning to handle this for SCM block deletion and other OM services in another PR? In that case maybe we should file the PRs under sub-jiras of the original HDDS-11508 Jira for easier tracking.

[sadanand48]

Are you planning to handle this for SCM block deletion and other OM services

Sure, will create sub tasks for these when the current one is fixed.

[errose28]

Sounds good, I think that will help with reviews too. In that case, let's use a new Jira for this PR and resolve HDDS-11508 once all the subtasks and their PRs are complete.

[adoroszlai]

@sadanand48 HDDS-11605 introduced some conflicts, can you please update the PR?

[sumitagrawl]

@sadanand48 Thanks for working over this, have few minor comments...

[sadanand48]

remove the limit case and use only ratis size.

@sumitagrawl @errose28 Could you please review this ? I have implemented the above suggestion and only used ratis size as the limit for sending a delete request.