In order to delete the client, you need to go to Clients → Select the desired client → Click on the trashcan icon.
NOTE: When deleting a client, all the projects and time entries will be deleted as well. The invoices, estimates, expenses, and reports associated with that client won’t be deleted.
If you wish to keep all their data, a better alternative would be to archive them.