cosmic-files/src/operation/controller.rs

109 lines
2.3 KiB
Rust
Raw Normal View History

use crate::fl;
feat: use io_uring / IOCP when available for async file IO (#911) Spawns a single thread for handling async file IO on the [compio runtime](https://github.com/compio-rs/compio). It is a completion-based IO runtime that can dynamically select a polling mechanism at runtime. It defaults to io_uring on Linux, IOCP on Windows, and the polling crate everywhere else. On Linux systems where io_uring is unavailable or disabled, it will fall back to the polling crate. This eliminates most of the threads that were needed previously. It significantly reduced the amount of memory needed in the recursive Context to get a good transfer rate for each copy operation—from a 4 MB buffer to 128 KB. Copies on a nvme drive are somewhat faster with the async IO changes, and use less CPU than before. Although it uses a single thread for non-blocking tasks, it still manages to 100% max out my nvme drive's activity for the whole duration of multiple long transfers. But it would be possible to enable compio's dispatcher to spread operations across worker threads if necessary. All but the extract and compress operations were updated to be async. I had to switch the `CondVar` in the `Controller` to a `tokio::sync::Notify` to prevent the IO thread from being put to sleep when an operation is paused. Fixed a deadlock in the `operation_copy` test function that was performing an operation without concurrently pulling from the channel in the operation. Reduced the rate that `Message::None` is sent from a subscription to trigger a UI redraw, and fixed it to not run when operations are paused.
2025-04-09 23:15:07 +02:00
use std::sync::{Arc, Mutex};
use tokio::sync::Notify;
#[derive(Clone, Copy, Debug)]
pub enum ControllerState {
Cancelled,
Paused,
Running,
}
#[derive(Debug)]
struct ControllerInner {
state: Mutex<ControllerState>,
progress: Mutex<f32>,
feat: use io_uring / IOCP when available for async file IO (#911) Spawns a single thread for handling async file IO on the [compio runtime](https://github.com/compio-rs/compio). It is a completion-based IO runtime that can dynamically select a polling mechanism at runtime. It defaults to io_uring on Linux, IOCP on Windows, and the polling crate everywhere else. On Linux systems where io_uring is unavailable or disabled, it will fall back to the polling crate. This eliminates most of the threads that were needed previously. It significantly reduced the amount of memory needed in the recursive Context to get a good transfer rate for each copy operation—from a 4 MB buffer to 128 KB. Copies on a nvme drive are somewhat faster with the async IO changes, and use less CPU than before. Although it uses a single thread for non-blocking tasks, it still manages to 100% max out my nvme drive's activity for the whole duration of multiple long transfers. But it would be possible to enable compio's dispatcher to spread operations across worker threads if necessary. All but the extract and compress operations were updated to be async. I had to switch the `CondVar` in the `Controller` to a `tokio::sync::Notify` to prevent the IO thread from being put to sleep when an operation is paused. Fixed a deadlock in the `operation_copy` test function that was performing an operation without concurrently pulling from the channel in the operation. Reduced the rate that `Message::None` is sent from a subscription to trigger a UI redraw, and fixed it to not run when operations are paused.
2025-04-09 23:15:07 +02:00
notify: Notify,
}
#[derive(Debug)]
pub struct Controller {
primary: bool,
inner: Arc<ControllerInner>,
}
impl Default for Controller {
fn default() -> Self {
Self {
primary: true,
inner: Arc::new(ControllerInner {
state: Mutex::new(ControllerState::Running),
progress: Mutex::new(0.0),
feat: use io_uring / IOCP when available for async file IO (#911) Spawns a single thread for handling async file IO on the [compio runtime](https://github.com/compio-rs/compio). It is a completion-based IO runtime that can dynamically select a polling mechanism at runtime. It defaults to io_uring on Linux, IOCP on Windows, and the polling crate everywhere else. On Linux systems where io_uring is unavailable or disabled, it will fall back to the polling crate. This eliminates most of the threads that were needed previously. It significantly reduced the amount of memory needed in the recursive Context to get a good transfer rate for each copy operation—from a 4 MB buffer to 128 KB. Copies on a nvme drive are somewhat faster with the async IO changes, and use less CPU than before. Although it uses a single thread for non-blocking tasks, it still manages to 100% max out my nvme drive's activity for the whole duration of multiple long transfers. But it would be possible to enable compio's dispatcher to spread operations across worker threads if necessary. All but the extract and compress operations were updated to be async. I had to switch the `CondVar` in the `Controller` to a `tokio::sync::Notify` to prevent the IO thread from being put to sleep when an operation is paused. Fixed a deadlock in the `operation_copy` test function that was performing an operation without concurrently pulling from the channel in the operation. Reduced the rate that `Message::None` is sent from a subscription to trigger a UI redraw, and fixed it to not run when operations are paused.
2025-04-09 23:15:07 +02:00
notify: Notify::new(),
}),
}
}
}
impl Controller {
feat: use io_uring / IOCP when available for async file IO (#911) Spawns a single thread for handling async file IO on the [compio runtime](https://github.com/compio-rs/compio). It is a completion-based IO runtime that can dynamically select a polling mechanism at runtime. It defaults to io_uring on Linux, IOCP on Windows, and the polling crate everywhere else. On Linux systems where io_uring is unavailable or disabled, it will fall back to the polling crate. This eliminates most of the threads that were needed previously. It significantly reduced the amount of memory needed in the recursive Context to get a good transfer rate for each copy operation—from a 4 MB buffer to 128 KB. Copies on a nvme drive are somewhat faster with the async IO changes, and use less CPU than before. Although it uses a single thread for non-blocking tasks, it still manages to 100% max out my nvme drive's activity for the whole duration of multiple long transfers. But it would be possible to enable compio's dispatcher to spread operations across worker threads if necessary. All but the extract and compress operations were updated to be async. I had to switch the `CondVar` in the `Controller` to a `tokio::sync::Notify` to prevent the IO thread from being put to sleep when an operation is paused. Fixed a deadlock in the `operation_copy` test function that was performing an operation without concurrently pulling from the channel in the operation. Reduced the rate that `Message::None` is sent from a subscription to trigger a UI redraw, and fixed it to not run when operations are paused.
2025-04-09 23:15:07 +02:00
pub async fn check(&self) -> Result<(), String> {
loop {
feat: use io_uring / IOCP when available for async file IO (#911) Spawns a single thread for handling async file IO on the [compio runtime](https://github.com/compio-rs/compio). It is a completion-based IO runtime that can dynamically select a polling mechanism at runtime. It defaults to io_uring on Linux, IOCP on Windows, and the polling crate everywhere else. On Linux systems where io_uring is unavailable or disabled, it will fall back to the polling crate. This eliminates most of the threads that were needed previously. It significantly reduced the amount of memory needed in the recursive Context to get a good transfer rate for each copy operation—from a 4 MB buffer to 128 KB. Copies on a nvme drive are somewhat faster with the async IO changes, and use less CPU than before. Although it uses a single thread for non-blocking tasks, it still manages to 100% max out my nvme drive's activity for the whole duration of multiple long transfers. But it would be possible to enable compio's dispatcher to spread operations across worker threads if necessary. All but the extract and compress operations were updated to be async. I had to switch the `CondVar` in the `Controller` to a `tokio::sync::Notify` to prevent the IO thread from being put to sleep when an operation is paused. Fixed a deadlock in the `operation_copy` test function that was performing an operation without concurrently pulling from the channel in the operation. Reduced the rate that `Message::None` is sent from a subscription to trigger a UI redraw, and fixed it to not run when operations are paused.
2025-04-09 23:15:07 +02:00
match self.state() {
ControllerState::Cancelled => return Err(fl!("cancelled")),
feat: use io_uring / IOCP when available for async file IO (#911) Spawns a single thread for handling async file IO on the [compio runtime](https://github.com/compio-rs/compio). It is a completion-based IO runtime that can dynamically select a polling mechanism at runtime. It defaults to io_uring on Linux, IOCP on Windows, and the polling crate everywhere else. On Linux systems where io_uring is unavailable or disabled, it will fall back to the polling crate. This eliminates most of the threads that were needed previously. It significantly reduced the amount of memory needed in the recursive Context to get a good transfer rate for each copy operation—from a 4 MB buffer to 128 KB. Copies on a nvme drive are somewhat faster with the async IO changes, and use less CPU than before. Although it uses a single thread for non-blocking tasks, it still manages to 100% max out my nvme drive's activity for the whole duration of multiple long transfers. But it would be possible to enable compio's dispatcher to spread operations across worker threads if necessary. All but the extract and compress operations were updated to be async. I had to switch the `CondVar` in the `Controller` to a `tokio::sync::Notify` to prevent the IO thread from being put to sleep when an operation is paused. Fixed a deadlock in the `operation_copy` test function that was performing an operation without concurrently pulling from the channel in the operation. Reduced the rate that `Message::None` is sent from a subscription to trigger a UI redraw, and fixed it to not run when operations are paused.
2025-04-09 23:15:07 +02:00
ControllerState::Paused => (),
ControllerState::Running => return Ok(()),
}
feat: use io_uring / IOCP when available for async file IO (#911) Spawns a single thread for handling async file IO on the [compio runtime](https://github.com/compio-rs/compio). It is a completion-based IO runtime that can dynamically select a polling mechanism at runtime. It defaults to io_uring on Linux, IOCP on Windows, and the polling crate everywhere else. On Linux systems where io_uring is unavailable or disabled, it will fall back to the polling crate. This eliminates most of the threads that were needed previously. It significantly reduced the amount of memory needed in the recursive Context to get a good transfer rate for each copy operation—from a 4 MB buffer to 128 KB. Copies on a nvme drive are somewhat faster with the async IO changes, and use less CPU than before. Although it uses a single thread for non-blocking tasks, it still manages to 100% max out my nvme drive's activity for the whole duration of multiple long transfers. But it would be possible to enable compio's dispatcher to spread operations across worker threads if necessary. All but the extract and compress operations were updated to be async. I had to switch the `CondVar` in the `Controller` to a `tokio::sync::Notify` to prevent the IO thread from being put to sleep when an operation is paused. Fixed a deadlock in the `operation_copy` test function that was performing an operation without concurrently pulling from the channel in the operation. Reduced the rate that `Message::None` is sent from a subscription to trigger a UI redraw, and fixed it to not run when operations are paused.
2025-04-09 23:15:07 +02:00
self.inner.notify.notified().await;
}
}
pub fn progress(&self) -> f32 {
*self.inner.progress.lock().unwrap()
}
pub fn set_progress(&self, progress: f32) {
*self.inner.progress.lock().unwrap() = progress;
}
pub fn state(&self) -> ControllerState {
*self.inner.state.lock().unwrap()
}
pub fn set_state(&self, state: ControllerState) {
*self.inner.state.lock().unwrap() = state;
feat: use io_uring / IOCP when available for async file IO (#911) Spawns a single thread for handling async file IO on the [compio runtime](https://github.com/compio-rs/compio). It is a completion-based IO runtime that can dynamically select a polling mechanism at runtime. It defaults to io_uring on Linux, IOCP on Windows, and the polling crate everywhere else. On Linux systems where io_uring is unavailable or disabled, it will fall back to the polling crate. This eliminates most of the threads that were needed previously. It significantly reduced the amount of memory needed in the recursive Context to get a good transfer rate for each copy operation—from a 4 MB buffer to 128 KB. Copies on a nvme drive are somewhat faster with the async IO changes, and use less CPU than before. Although it uses a single thread for non-blocking tasks, it still manages to 100% max out my nvme drive's activity for the whole duration of multiple long transfers. But it would be possible to enable compio's dispatcher to spread operations across worker threads if necessary. All but the extract and compress operations were updated to be async. I had to switch the `CondVar` in the `Controller` to a `tokio::sync::Notify` to prevent the IO thread from being put to sleep when an operation is paused. Fixed a deadlock in the `operation_copy` test function that was performing an operation without concurrently pulling from the channel in the operation. Reduced the rate that `Message::None` is sent from a subscription to trigger a UI redraw, and fixed it to not run when operations are paused.
2025-04-09 23:15:07 +02:00
self.inner.notify.notify_waiters();
}
pub fn is_cancelled(&self) -> bool {
matches!(self.state(), ControllerState::Cancelled)
}
pub fn cancel(&self) {
self.set_state(ControllerState::Cancelled);
}
pub fn is_paused(&self) -> bool {
matches!(self.state(), ControllerState::Paused)
}
pub fn pause(&self) {
self.set_state(ControllerState::Paused);
}
pub fn unpause(&self) {
feat: use io_uring / IOCP when available for async file IO (#911) Spawns a single thread for handling async file IO on the [compio runtime](https://github.com/compio-rs/compio). It is a completion-based IO runtime that can dynamically select a polling mechanism at runtime. It defaults to io_uring on Linux, IOCP on Windows, and the polling crate everywhere else. On Linux systems where io_uring is unavailable or disabled, it will fall back to the polling crate. This eliminates most of the threads that were needed previously. It significantly reduced the amount of memory needed in the recursive Context to get a good transfer rate for each copy operation—from a 4 MB buffer to 128 KB. Copies on a nvme drive are somewhat faster with the async IO changes, and use less CPU than before. Although it uses a single thread for non-blocking tasks, it still manages to 100% max out my nvme drive's activity for the whole duration of multiple long transfers. But it would be possible to enable compio's dispatcher to spread operations across worker threads if necessary. All but the extract and compress operations were updated to be async. I had to switch the `CondVar` in the `Controller` to a `tokio::sync::Notify` to prevent the IO thread from being put to sleep when an operation is paused. Fixed a deadlock in the `operation_copy` test function that was performing an operation without concurrently pulling from the channel in the operation. Reduced the rate that `Message::None` is sent from a subscription to trigger a UI redraw, and fixed it to not run when operations are paused.
2025-04-09 23:15:07 +02:00
if !self.is_cancelled() {
self.set_state(ControllerState::Running);
}
}
}
impl Clone for Controller {
fn clone(&self) -> Self {
Self {
primary: false,
inner: self.inner.clone(),
}
}
}
impl Drop for Controller {
fn drop(&mut self) {
// Cancel operations if primary controller is dropped
if self.primary {
self.cancel();
}
}
}