Add a proper VFS in a struct which now lives on the heap with no hacky

static lifetimes, move cursor.bmp to image assets and add wallpapers,
and a logo, add Github Action that auto-releases, add build_all script
which just builds without running

.github/workflows/release.yml

@@ -0,0 +1,86 @@
+name: Build & Release XunilOS
+
+on:
+  push:
+    branches:
+      - main
+
+jobs:
+  build:
+    name: Build (${{ matrix.karch }})
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        karch: [x86_64, aarch64]
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Install build dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y \
+            build-essential \
+            gcc \
+            gcc-aarch64-linux-gnu \
+            binutils-aarch64-linux-gnu \
+            nasm \
+            xorriso \
+            grub-pc-bin \
+            grub-efi-amd64-bin \
+            grub-efi-arm64-bin \
+            mtools \
+            curl
+
+      - name: Install Rust toolchain
+        uses: dtolnay/rust-toolchain@stable
+        with:
+          targets: |
+            x86_64-unknown-none
+            aarch64-unknown-none
+
+      - name: Run build script
+        run: bash build_all.sh
+        env:
+          KARCH: ${{ matrix.karch }}
+
+      - name: Upload ISO artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: XunilOS-${{ matrix.karch }}.iso
+          path: XunilOS-${{ matrix.karch }}.iso
+          if-no-files-found: error
+
+  release:
+    name: Publish Release
+    runs-on: ubuntu-latest
+    needs: build
+    permissions:
+      contents: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Download x86_64 ISO
+        uses: actions/download-artifact@v4
+        with:
+          name: XunilOS-x86_64.iso
+
+      - name: Download aarch64 ISO
+        uses: actions/download-artifact@v4
+        with:
+          name: XunilOS-aarch64.iso
+
+      - name: Create GitHub Release
+        uses: softprops/action-gh-release@v2
+        with:
+          tag_name: commit-${{ github.sha }}
+          name: Commit ${{ github.sha }}
+          body: |
+            Automated release for commit [${{ github.sha }}](${{ github.server_url }}/${{ github.repository }}/commit/${{ github.sha }})
+          files: |
+            XunilOS-x86_64.iso
+            XunilOS-aarch64.iso
+          fail_on_unmatched_files: true

README.md

@@ -1,34 +1,46 @@
 # XunilOS
 XunilOS is an OS made from scratch in Rust.
-
 The repo is based on the limine-rust-template.
 
-## How to use this?
+It supports aarch64 inside QEMU, and x86_64 even on bare machines!
 
-### Dependencies
+# Features
 
-Any `make` command depends on GNU make (`gmake`) and is expected to be run using it. This usually means using `make` on most GNU/Linux distros, or `gmake` on other non-GNU systems.
+## Kernel
+- x86_64 IDT, GDT, interrupts, kernel heap, PS2 mouse/keyboard, paging, syscalls and usermode.
+- aarch64 kernel heap, paging, interrupts, Virtio mouse/keyboard, syscalls, usermode
+- Scheduler which does round-robin switching as well as sleeping and waking processes
+- ELF64 ET_EXEC and ET_DYN loading, verifying and running support
+- A readonly VFS which currently includes the ELF files which can be ran
+- IPC with granular permissions (read, write, manage)
+- basic (and insecure) SHM support
+- Limine bootloader
+- Framebuffer and serial support
+- Timing support based on IRQ
+- Per-process address space, kernel & user stack
+- Copy to and from userspace
 
-All `make all*` targets depend on Rust.
+## Apps
+- doomgeneric: Doom ported to XunilOS. Isn't as easy, as i am using Rust and had to write my own libc stub.
+- badapple: Grayscale Bad Apple by using numbers to represent shades of gray. Pretty easy, but this is the only place where I also used Python.
+- shell: simple shell with elf running, echo and file read commands.
+- helloworld: just prints helloworld to serial
 
-Additionally, building an ISO with `make all` requires `xorriso`, and building a HDD/USB image with `make all-hdd` requires `sgdisk` (usually from `gdisk` or `gptfdisk` packages) and `mtools`.
+## Init
+- Desktop-like experience
+- Window Management (close and minimize)
+- Start menu to open applications
+- BMP background
+- Mouse support with a BMP image
+- Dock where you can see currently open applications which can be minimized or unminimized
 
-### Architectural targets
-
-The `KARCH` make variable determines the target architecture to build the kernel and image for.
-
-The default `KARCH` is `x86_64`. Other options include: `aarch64`, `riscv64`, and `loongarch64`.
-
-Other architectures will need to be enabled in kernel/rust-toolchain.toml
-
-### Makefile targets
-
-Running `make all` will compile the kernel (from the `kernel/` directory) and then generate a bootable ISO image.
-
-Running `make all-hdd` will compile the kernel and then generate a raw image suitable to be flashed onto a USB stick or hard drive/SSD.
-
-Running `make run` will build the kernel and a bootable ISO (equivalent to make all) and then run it using `qemu` (if installed).
-
-Running `make run-hdd` will build the kernel and a raw HDD image (equivalent to make all-hdd) and then run it using `qemu` (if installed).
-
-The `run-uefi` and `run-hdd-uefi` targets are equivalent to their non `-uefi` counterparts except that they boot `qemu` using a UEFI-compatible firmware.
+## Libxunil (libc stub)
+- Basic functions of C (printf, strlen, etc)
+- File I\O, IPC, time
+- Input reading from Window Management
+- SHM
+- User Heap
+- Window management
+- IPC support
+- Syscalls to call back to the kernel
+- Primitives, Framebuffer and font rendering

build_all.sh

@@ -0,0 +1,7 @@
+bash build_rust_app.sh libxunil
+bash build_rust_app.sh init
+bash build_doomgeneric.sh
+bash build_helloworld.sh
+bash build_rust_app.sh badapple
+bash build_rust_app.sh shell
+make all

build_and_run.sh

@@ -1,8 +1,3 @@
-export KARCH=x86_64
-bash build_rust_app.sh libxunil
-bash build_rust_app.sh init
-bash build_doomgeneric.sh
-bash build_helloworld.sh
-bash build_rust_app.sh badapple
-bash build_rust_app.sh shell
+export KARCH=aarch64
+bash build_all.sh
 make run

kernel/src/arch/aarch64/init.rs

@@ -7,7 +7,10 @@ use crate::{
         },
         arch::KERNEL_MAPPER,
     },
-    driver::{io::virtio::scan_virtio_devices, ipc::init_ipc},
+    driver::{
+        io::{fs::vfs::init_vfs, virtio::scan_virtio_devices},
+        ipc::init_ipc,
+    },
     mm::shm::init_shm,
 };
 use limine::response::{ExecutableAddressResponse, HhdmResponse, MemoryMapResponse};
@@ -40,6 +43,7 @@ pub extern "C" fn init_aarch64(mapper: &mut AArchPageTable) {
     init_interrupts();
     init_ipc();
     init_shm();
+    init_vfs();
 }
 
 pub fn preinit_aarch64<'a>(

kernel/src/arch/syscall.rs

@@ -4,9 +4,11 @@ use core::sync::atomic::Ordering;
 
 #[cfg(target_arch = "x86_64")]
 use crate::arch::x86_64::paging::create_and_map_multiple_pages;
+use crate::driver::io::fs::vfs::vfs_write;
 use crate::driver::io::input::{InputEvent, process_input};
 #[cfg(target_arch = "x86_64")]
 use crate::driver::io::ps2::process_scancodes;
+use crate::mm::usercopy::copy_from_user;
 use alloc::vec;
 use alloc::{string::String, vec::Vec};
 #[cfg(target_arch = "x86_64")]
@@ -204,6 +206,39 @@ fn open(path: isize, mode: isize) -> isize {
         .unwrap_or(-1)
 }
 
+fn write_fd(ptr: isize, size: isize, count: isize, fd: isize) -> isize {
+    let pid = current_pid().unwrap_or(0);
+    if pid == 0 {
+        return -1;
+    }
+
+    SCHEDULER
+        .with_process(pid, |process| {
+            let len = (size as usize).checked_mul(count as usize).ok_or(-1isize)?;
+            if len == 0 {
+                return Ok(0isize);
+            }
+
+            let address_space = process.address_space.as_mut().ok_or(-1isize)?;
+
+            let mut buf: alloc::vec::Vec<u8> = alloc::vec![0u8; len];
+            copy_from_user(
+                &mut address_space.mapper,
+                buf.as_mut_ptr(),
+                ptr as *const u8,
+                len,
+            )
+            .map_err(|_| -14isize)?;
+
+            Ok(
+                vfs_write(buf.as_ptr(), size as usize, count as usize, fd as i64).unwrap_or(0)
+                    as isize,
+            )
+        })
+        .unwrap_or(Err(-1))
+        .unwrap_or(-1)
+}
+
 fn close(fd: isize) -> isize {
     vfs_close(fd as i64) as isize
 }
@@ -656,18 +691,22 @@ pub unsafe extern "C" fn syscall_dispatch(
         BRK => unsafe { sbrk(arg0) },
         READ => read(arg0, arg1, arg2, arg3) as isize,
         WRITE => {
-            let buf_ptr = arg1 as *const u8;
-            let len = arg2 as usize;
-            let bytes: &[u8] = unsafe { core::slice::from_raw_parts(buf_ptr, len) };
-            if let Ok(s) = core::str::from_utf8(bytes) {
-                print!("{}", s);
-            } else {
-                for byte in bytes {
-                    if *byte == b'\0' {
-                        continue;
+            if arg0 == 1 {
+                let buf_ptr = arg1 as *const u8;
+                let len = arg2 as usize;
+                let bytes: &[u8] = unsafe { core::slice::from_raw_parts(buf_ptr, len) };
+                if let Ok(s) = core::str::from_utf8(bytes) {
+                    print!("{}", s);
+                } else {
+                    for byte in bytes {
+                        if *byte == b'\0' {
+                            continue;
+                        }
+                        print!("{}", *byte as char);
                     }
-                    print!("{}", *byte as char);
                 }
+            } else {
+                write_fd(arg0, arg1, arg2, arg3);
             }
 
             0
@@ -678,7 +717,9 @@ pub unsafe extern "C" fn syscall_dispatch(
         EXIT => kill(current_pid().unwrap() as isize, arg0),
         SLEEP => sleep(arg0),
         EXECVE => exec(arg0),
-        CLOCK_GETTIME => ((TIMER.now().elapsed() as usize) * (TIMER_FREQUENCY_HZ / 1000)) as isize,
+        CLOCK_GETTIME => {
+            (TIMER.now().elapsed() + TIMER.get_date_at_boot() * TIMER_FREQUENCY_HZ as u64) as isize
+        }
         MAP_FRAMEBUFFER => map_framebuffer(),
         INPUT_READ => input_read(arg0 as *mut InputEvent, arg1),
         GETPID => {

kernel/src/arch/x86_64/init.rs

@@ -8,7 +8,7 @@ use crate::{
         syscall::init_syscalls,
     },
     config::TIMER_FREQUENCY_HZ,
-    driver::ipc::init_ipc,
+    driver::{io::fs::vfs::init_vfs, ipc::init_ipc},
     mm::shm::init_shm,
 };
 
@@ -93,6 +93,7 @@ pub fn init_x86_64<'a>(
 
     init_ipc();
     init_shm();
+    init_vfs();
 
     return mapper;
 }

kernel/src/driver/io/fs/vfs.rs

@@ -1,214 +1,233 @@
+use alloc::{
+    collections::btree_map::BTreeMap,
+    string::{String, ToString},
+    vec::Vec,
+};
+use lazy_static::lazy_static;
+
 use crate::driver::io::fs::assets::*;
-use core::ptr::{null, null_mut};
+
+lazy_static! {
+    static ref FILE_CONTENT: BTreeMap<&'static str, &'static [u8]> = {
+        let mut map = BTreeMap::new();
+        map.insert("testfile", &b"Hello, World!"[..]);
+        map.insert("helloworld.elf", HELLOWORLD_ELF);
+        map.insert("badapple", BADAPPLE_ELF);
+        map.insert("doomgeneric", DOOM_ELF);
+        map.insert("shell", SHELL_ELF);
+        map.insert("doom1.wad", DOOM_WAD);
+        map.insert("doom.cfg", &b""[..]);
+        map.insert("default.cfg", &b""[..]);
+        map
+    };
+}
 
 #[repr(C)]
-#[derive(Clone, Copy)]
+#[derive(Clone, Debug)]
 pub struct FILE {
-    pub data: *const u8,    // pointer to the file's data
-    pub size: usize,        // total size
-    pub cursor: usize,      // current position
-    pub writable: bool,     // is this a write buffer?
-    pub write_buf: *mut u8, // for writable fake files
-    pub write_cap: usize,
+    pub name: String,
+    pub size: usize,
+    pub data: Vec<u8>,
+    pub cursor: usize,
+    pub writable: bool,
     pub fd: i64,
 }
 
 impl FILE {
-    pub const fn zeroed() -> FILE {
+    pub fn new(name: String, data: Vec<u8>, writable: bool) -> FILE {
         FILE {
-            data: null(),
-            size: 0,
+            name,
+            data: data.clone(),
             cursor: 0,
-            writable: false,
-            write_buf: null_mut(),
-            write_cap: 0,
+            writable,
             fd: -1,
+            size: data.len(),
         }
     }
 }
 
-struct FakeFileEntry {
-    name: &'static str,
-    data: &'static [u8],
+pub struct VFS {
+    files: Vec<FILE>,
+    next_fd: i64,
 }
 
-pub type Fd = i64;
-const MAX_FD: usize = 16;
-
-fn fd_ok(fd: Fd) -> bool {
-    fd >= 0 && (fd as usize) < MAX_FD
-}
-
-static FILES: &[FakeFileEntry] = &[
-    FakeFileEntry {
-        name: "testfile",
-        data: b"Hello, World!",
-    },
-    FakeFileEntry {
-        name: "helloworld.elf",
-        data: HELLOWORLD_ELF,
-    },
-    FakeFileEntry {
-        name: "badapple",
-        data: BADAPPLE_ELF,
-    },
-    FakeFileEntry {
-        name: "doomgeneric",
-        data: DOOM_ELF,
-    },
-    FakeFileEntry {
-        name: "shell",
-        data: SHELL_ELF,
-    },
-    FakeFileEntry {
-        name: "doom1.wad",
-        data: DOOM_WAD,
-    },
-    FakeFileEntry {
-        name: "default.cfg",
-        data: b"",
-    },
-    FakeFileEntry {
-        name: "doom.cfg",
-        data: b"",
-    },
-];
-
-static mut FILE_POOL: [FILE; 16] = [FILE::zeroed(); 16];
-static mut FILE_POOL_USED: [bool; 16] = [false; 16];
-
-pub unsafe fn get_file_pool_slot() -> (*mut FILE, i64) {
-    unsafe {
-        for i in 0..16 {
-            if !FILE_POOL_USED[i] {
-                FILE_POOL_USED[i] = true;
-                return (&mut FILE_POOL[i], i as i64);
-            }
-        }
-
-        (null_mut(), -1)
-    }
-}
-
-unsafe fn file_mut(fd: Fd) -> Option<&'static mut FILE> {
-    if !fd_ok(fd) {
-        return None;
-    }
-    let idx = fd as usize;
-
-    if unsafe { !FILE_POOL_USED[idx] } {
-        return None;
-    }
-
-    return unsafe { Some(&mut FILE_POOL[idx]) };
-}
-
-#[unsafe(no_mangle)]
-pub fn vfs_open(name: &str, _mode: &str) -> Fd {
-    for entry in FILES {
-        if entry.name.contains(name) {
-            let (slot, fd) = unsafe { get_file_pool_slot() };
-            if slot.is_null() {
-                return -1;
-            }
-
-            unsafe {
-                (*slot).data = entry.data.as_ptr();
-                (*slot).size = entry.data.len();
-                (*slot).cursor = 0;
-                (*slot).writable = false;
-                (*slot).write_buf = null_mut();
-                (*slot).write_cap = 0;
-                (*slot).fd = fd;
-            }
-            return fd;
+impl VFS {
+    pub fn new() -> VFS {
+        VFS {
+            files: Vec::new(),
+            next_fd: 0,
         }
     }
-    -1
-}
 
-#[unsafe(no_mangle)]
-pub fn vfs_close(fd: Fd) -> i32 {
-    if !fd_ok(fd) {
-        return -1;
+    pub fn open(&mut self, name: &str, mode: &str) -> i64 {
+        let is_write = mode.contains("w") || mode.contains("a");
+
+        if let Some(file) = self
+            .files
+            .iter_mut()
+            .find(|file| file.name.as_str() == name)
+        {
+            file.cursor = 0;
+            file.writable = is_write;
+
+            return file.fd;
+        }
+
+        let fd = self.next_fd;
+        self.next_fd += 1;
+
+        let empty_data = &"".as_bytes();
+
+        let data = FILE_CONTENT.get(name).unwrap_or(empty_data);
+
+        let file = FILE {
+            name: name.to_string(),
+            data: data.to_vec(),
+            cursor: 0,
+            writable: is_write,
+            fd,
+            size: data.len(),
+        };
+
+        self.files.push(file);
+
+        fd
     }
 
-    unsafe {
-        let idx = fd as usize;
-        if !FILE_POOL_USED[idx] {
+    pub fn close(&mut self, fd: i64) -> i32 {
+        if let Some(file_pos) = self.files.iter().position(|file| file.fd == fd) {
+            self.files.remove(file_pos);
+            0
+        } else {
+            -1
+        }
+    }
+
+    pub fn lseek(&mut self, fd: i64, offset: i64, whence: i32) -> i64 {
+        let f = match self.files.iter_mut().find(|file| file.fd == fd) {
+            Some(f) => f,
+            None => return -1,
+        };
+
+        let new_pos = match whence {
+            0 => {
+                if offset < 0 {
+                    return -1;
+                }
+                offset as usize
+            }
+            1 => {
+                let cur = f.cursor as i64;
+                let pos = cur.saturating_add(offset);
+                if pos < 0 {
+                    return -1;
+                }
+                pos as usize
+            }
+            2 => {
+                let end = f.size as i64;
+                let pos = end.saturating_add(offset);
+                if pos < 0 {
+                    return -1;
+                }
+                pos as usize
+            }
+            _ => return -1,
+        };
+
+        if new_pos > f.size {
             return -1;
         }
-        FILE_POOL_USED[idx] = false;
-        FILE_POOL[idx] = FILE::zeroed();
+
+        f.cursor = new_pos;
+        f.cursor as i64
     }
 
-    0
-}
+    pub fn read(&mut self, fd: i64, len: usize) -> Option<(*const u8, usize)> {
+        if let Some(f) = self.files.iter_mut().find(|file| file.fd == fd) {
+            if f.cursor > f.size {
+                return Some((f.data.as_ptr(), 0));
+            }
 
-#[unsafe(no_mangle)]
-#[allow(unused_variables)]
-pub fn vfs_write(ptr: *mut u8, size: usize, count: usize, fp: *mut FILE) -> usize {
-    if ptr.is_null() || fp.is_null() || unsafe { (*fp).fd < 0 || (*fp).fd >= 16 } {
-        return 0;
+            let available = f.size - f.cursor;
+            let to_read = len.min(available);
+
+            let src = unsafe { f.data.as_ptr().add(f.cursor) };
+            f.cursor = f.cursor.saturating_add(to_read);
+
+            Some((src, to_read))
+        } else {
+            None
+        }
+    }
+
+    pub fn write(&mut self, fd: i64, data: &[u8]) -> Option<usize> {
+        let file = self.files.iter_mut().find(|f| f.fd == fd)?;
+
+        if !file.writable {
+            return None;
+        }
+
+        file.data.extend_from_slice(data);
+        file.size = file.data.len();
+        Some(data.len())
     }
-    count
 }
 
-#[unsafe(no_mangle)]
-pub fn vfs_read(fd: Fd, len: usize) -> Option<(*const u8, usize)> {
+pub static mut VFS_INSTANCE: Option<VFS> = None;
+
+pub fn init_vfs() {
     unsafe {
-        let f = file_mut(fd)?;
-        if f.cursor > f.size {
-            return Some((f.data, 0));
-        }
-
-        let available = f.size - f.cursor;
-        let to_read = len.min(available);
-
-        let src = f.data.add(f.cursor);
-        f.cursor = f.cursor.saturating_add(to_read);
-
-        Some((src, to_read))
+        VFS_INSTANCE = Some(VFS {
+            files: Vec::new(),
+            next_fd: 3,
+        })
     }
 }
 
 #[unsafe(no_mangle)]
-pub extern "C" fn vfs_lseek(fd: Fd, offset: i64, whence: i32) -> i64 {
-    let f = match unsafe { file_mut(fd) } {
-        Some(f) => f,
-        None => return -1,
-    };
+pub fn vfs_open(name: &str, mode: &str) -> i64 {
+    #[allow(static_mut_refs)]
+    unsafe {
+        VFS_INSTANCE.as_mut().unwrap().open(name, mode)
+    }
+}
 
-    let new_pos = match whence {
-        0 => {
-            if offset < 0 {
-                return -1;
-            }
-            offset as usize
-        }
-        1 => {
-            let cur = f.cursor as i64;
-            let pos = cur.saturating_add(offset);
-            if pos < 0 {
-                return -1;
-            }
-            pos as usize
-        }
-        2 => {
-            let end = f.size as i64;
-            let pos = end.saturating_add(offset);
-            if pos < 0 {
-                return -1;
-            }
-            pos as usize
-        }
-        _ => return -1,
-    };
+#[unsafe(no_mangle)]
+pub fn vfs_close(fd: i64) -> i32 {
+    #[allow(static_mut_refs)]
+    unsafe {
+        VFS_INSTANCE.as_mut().unwrap().close(fd)
+    }
+}
 
-    if new_pos > f.size {
-        return -1;
+#[unsafe(no_mangle)]
+pub fn vfs_write(ptr: *const u8, size: usize, count: usize, fd: i64) -> Option<usize> {
+    if ptr.is_null() {
+        return None;
     }
 
-    f.cursor = new_pos;
-    f.cursor as i64
+    let len = size.checked_mul(count)?;
+
+    unsafe {
+        let slice = core::slice::from_raw_parts(ptr, len);
+        #[allow(static_mut_refs)]
+        VFS_INSTANCE.as_mut().unwrap().write(fd, slice)
+    }
+}
+
+#[unsafe(no_mangle)]
+pub fn vfs_read(fd: i64, len: usize) -> Option<(*const u8, usize)> {
+    #[allow(static_mut_refs)]
+    unsafe {
+        VFS_INSTANCE.as_mut().unwrap().read(fd, len)
+    }
+}
+
+#[unsafe(no_mangle)]
+pub extern "C" fn vfs_lseek(fd: i64, offset: i64, whence: i32) -> i64 {
+    #[allow(static_mut_refs)]
+    unsafe {
+        VFS_INSTANCE.as_mut().unwrap().lseek(fd, offset, whence)
+    }
 }