Al is probably too soft and flexible -- even cheap steel is more rigid.
Also, Al has an oxide layer. Al oxide is the grit in sandpaper. Sliding friction, especially metal-on-metal, will scrape small amounts off, resulting in a fine abrasive in the sliding parts. It'll certainly work for a while, and that could be a long while, long enough you may not care.