Message Passing Interface (MPI) provides an infrastructure that enables users to build a high performance distributed computing environment from networked computers with minimum effort. It provides a common Application Programming Interface (API) for the development of parallel applications regardless of the type of multiprocessor system used. This research implements a distributed computing system called Java Message Passing Interface Middleware which supports a Message Passing Interface Application Programming Interface (MPI API). It installs Java Message Passing Interface (JMPI) package and runs three applications (Range Addition, Matrix - Vector Multiplication and Gauss Elimination method) in two modes serial and parallel. The system implemented on a Local Area Network (LAN) consisted of five computers. Many experiments have been performed to test the system and it found that results of parallel applications were close to the results of serial applications because the calculation times of applications were simple compared to communication times.