The proliferation of Internet of Everything (IoE) and the success of rich Cloud services have pushed the horizon of a new computing paradigm, Edge Computing, which calls for processing the data at the edge of the network. Smart home as a typical IoE application is being widely adapted into peoples life. Edge Computing has the potential to empower the smart home, but it needs more contribution from the community before it truly benefits our lives. In this paper, we first present the vision of EdgeOSH, a home operating system for Internet of Everything, followed by the challenges in EdgeOSH, namely programming interface, self-management, data management, security & privacy, and naming. Within each challenge we also discuss the potential directions that are worth further investigation.