The growth of bacterial colony exhibits striking patterns that are determined by the interactions among individual, growing and dividing bacterial cells, and that between cells and the surrounding nutrient and waste. Understanding the principles that underlie such growth has far-reaching consequences in biological and health sciences. In this work, we construct a multi-scale model of the growth of E. coli cells on agar surface. Our model consists of detailed, microscopic descriptions of the cell growth, cell division with fluctuations, and cell movement due to the cell-cell and cell-environment mechanical interactions, and macroscopic diffusion equations for the nutrient and waste. Our large-scale simulations reproduce experimentally observed growth scaling laws, strip patterns, and many other features of an E. coli colony. This work is the first step toward detailed multi-scale computational modeling of three-dimensional bacterial growth with mechanical and chemical interactions.