Papers & Presentations - SEED Center for Data Farming
The Data Farming Book
Sanchez, S.M. and P.J. Sanchez (2025). Data Farming for Simulation Analysis, forthcoming.
Videos
Work Smarter, Not Harder: A Tutorial on Designing and Conducting Simulation Experiments video by Susan Sanchez, Paul Sanchez, and Hong Wan (North Carolina State University) presented Dec 14, 2021 at the 2021 Winter Simulation Conference, Phoenix, AZ. WARNING - THIS VIDEO IS 340 MB and 1 hour 12 minutes long.
WORMS and Work in the Time of Coronavirus, webinar by Susan Sanchez, presented May 15, 2020. WARNING - THIS VIDEO IS 500 MB and 60 minutes long. Click here for slides.
A Data Farmer's Almanac, "Titan Talk" video by Susan Sanchez, presented December 13, 2016 at the 2016 Winter Simulation Conference, Washington, DC. WARNING - THIS VIDEO IS 527 MB and 60 minutes long.
Introduction to Data Farming video by Paul Sanchez, presented May 8, 2010 at CENIC Conference, NPS, Monterey, CA. WARNING - THIS VIDEO IS 336 MB and 35 minutes long.
Papers & Presentations
You may request a copy of any paper without a direct link. We've organized things into categories for different audiences:
- Introductory materials, providing an overview of the philosophy and approaches we use when designing and analyzing simulation experiments;
- Selected articles and book chapters that describe methodological advances;
- Selected proceedings papers that contain a mix of theory and applications.
Introductory Papers
Sanchez, S. M., P.J. Sanchez, and H. Wan (2021). . Proceedings of the 2021 Winter Simulation Conference. Updated from earlier tutorials that appeared in 2018, 2015, 2012, 2011, 2009, ...,
Sanchez, S. M. (2020). ,&紳莉莽梯;
ACM Transactions on Modeling and Computer Simulation, 30(4): Article 22, 22:1-22:30.
Sanchez, S. M. (2018). . Chance, 2(31), 45-52.
Sanchez, S. M. (2018). . Proceedings of the 2018 Winter Simulation Conference, 425-439. Updated from other tutorials that appeared in 2014, 2015.
Sanchez, S. M. and P. J. Sanchez (2017). , Chapter 9 in Advances in Modeling and Simulation---Seminal Research from 50 Years of Winter Simulation Conferences, eds. A. Tolk, J. Fowler, G. Shao, and E. Yucesan, 159--179. Cham, Switzerland: Springer International Publishing.
Lucas, T.W., W.D. Kelton, P.J. Sanchez, S.M. Sanchez, and B.L. Anderson. 2015. Changing the Paradigm: Simulation, Now a Method of First Resort. Naval Research Logistics 62: 293305.
Sanchez, S.M., T.W. Lucas, P.J. Sanchez, C.J. Nannini, and H. Wan. 2012. Designing large scale simulation experiments, with applications to defense and homeland security. Chapter 12 in Design and Analysis of Experiments, Vol. 3: Special Designs and Applications, ed. K. Hinkelmann. Hoboken, NJ: Wiley.Winner, 2013 Koopman Prize, INFORMS Military Applications Society.
Lucas, T.W., S.M. Sanchez, F. Martinez, J.W. Roginski, L.R. Sickinger. 2007. Defense and homeland security applications of multi-agent simulations. Proceedings of the 2007 Winter Simulation Conference, 138149.
Kleijnen, J.P.C., S.M. Sanchez, T.W. Lucas, and T.M. Cioppa. 2005. A user's guide to the brave new world of designing simulation experiments. INFORMS Journal on Computing 17(3), 263289 (with online supplement).
More on Experimental Design and Analysis
Parker, J.D. and T.W. Lucas. (2024). Sequentially Extending Space-Filling Experimental Designs by Optimally Permuting and Stacking Columns of the Design Matrix, available online from July 2024 at the European Journal of Operational Research. URL: .
Lucas, T.W. and J.D. Parker. (2023). The Variability in Design Quality Measures for Multiple Types of Space-filling Designs Created by Leading Software Packages, Proceedings of the 2023 Winter Simulation Conference, 2023, 516-527. URL: .
Parker, J.D. (2022). Extending and improving designs for large-scale computer experiments, in: Doctoral dissertation, Monterey, CA; 窪蹋勛圖厙. URL: .
Erickson, C. B., B. E. Ankenman, M. Plumlee, and S. M. Sanchez (2021). , Quality and Reliability Engineering International, 37(7), 3084-3107 (Special Issue on Advancing Statistical Methods for Testing and Evaluating Defense Systems).
Morgan, L. E., A. C. Titman, R. A. Williams, and S. M. Sanchez (2021). . Proceedings of the Operational Research Society Simulation Workshop 21 (SW21), eds. M. Fakimi, D. Robertson, and T. Boness, pp. 127-136.
Sanchez, P. J., and S. M. Sanchez (2019). ,'' Quality and Reliability Engineering International, 35(3), 854-867 (Special Issue on Advancing Statistical Methods for Testing and Evaluating Defense Systems).
Sanchez, P. J., and Sanchez, S. M. (2018). Orthogonal 2nd order space-filling designs with insights from simulation experiments to support test planning. Quality and Reliability Engineering International, 35(3), 854-867.
Erickson, C., Ankenman, B. E., and Sanchez, S. M. (2018). . European Journal of Operational Research, 2018(266), 179-192.
Erickson, C., B. E. Ankenman, and S. M. Sanchez (2018). . Data in Brief, 18(June), 684-687.
Duan, W., B. E. Ankenman, P. J. Sanchez, and S. M. Sanchez (2017). . Technometrics, 59(1), 1122.
Erickson, C., B. E. Ankenman, and S. M. Sanchez (2016). . Extended abstract for poster session, Proceedings of the 2016 Winter Simulation Conference
MacCalman, A. D., H. Vieira, and T. W. Lucas (2016). Second-order nearly orthogonal Latin hypercubes for exploring stochastic simulations. Journal of Simulation, available online at .
Sanchez, S. M. (2015). "" Proceedings of the 2015 Winter Simulation Conference, 800-811. (Invited, expanded tutorial version of 2014 paper)
Sanchez, S. M. and H. Wan (2015). . Proceedings of the 2015 Winter Simulation Conference, 1795-1809. Updated from earlier tutorials that appeared in 2012, 2011, 2009, ...,
Sanchez, S. M. (2014). "" Proceedings of the 2014 Winter Simulation Conference, 805-806. (Invited, featured speaker in "Big Data" track)
Vieira Jr., H., S. M. Sanchez, P. J. Sanchez, K. H. Kienitz, M. C. M. Belderrain (2014). A restricted multinomial hybrid selection procedure, ACM Transactions on Modeling and Computer Simulation, 24(2), Article 10, pp. 10:110:24.
Sanchez, S. M., P. J. Sanchez, and H. Wan (2014). Simulation experiments: Better insights by design. Proceedings of the 2014 Summer Simulation Multi-Conference.
"Data farming in support of NATO." NATO Science and Technology Organization Technical Report TR-MSG-088. Available via . 226 pages. Winner, 2014 Scientific Achievement Award, NATO STO.
Vieira Jr., H., S. M. Sanchez, K. H. Kienitz, M. C. M. Belderrain (2013). Efficient, nearly orthogonal-and-balanced, mixed designs: An effective way to conduct trade-off analyses via simulation, Journal of Simulation 7: 264275 (Special Issue on Input/Output Analysis).
Hernandez, A., Lucas, T. W., and Carlyle, M. (2012). "Constructing nearly orthogonal Latin hypercubes for any nonsaturated run-variable combination." ACM Trans. Model. Comput. Simul. 22, 4, Article 20 (November 2012)
Sanchez, S. M. and H. Wan (2012). . " Proceedings of the 2012 Winter Simulation Conference, eds. C. Laroque, J. Himmelspach, R. Pasupathy, O. Rose, and A. M. Uhrmacher, Piscataway, New Jersey: Institute of Electrical and Electronic Engineers, Inc., 15521563. (Invited). Updated from earlier tutorials that appeared in 2011, 2009, ...,
Hernandez, A., Lucas, T. W. and Sanchez, P. J. (2012). Proceedings of the 2012 Winter Simulation Conference, eds. C. Laroque, J. Himmelspach, R. Pasupathy, O. Rose, and A. M. Uhrmacher, Piscataway, New Jersey: Institute of Electrical and Electronic Engineers, Inc.
Vieira Jr., H., S. Sanchez, K. H. Kienitz, and M. C. N. Belderrain (2011). Generating and improving orthogonal designs by using mixed integer programming, European Journal of Operational Research 215: 629638.
Sanchez, P. J., 2011, "Efficient generation of resolution VII fractional factorial designs," notes.
Sanchez, S. M. and H. Wan (2011). . Proceedings of the 2011 Winter Simulation Conference, 1441-1455.
Schruben, L. and D. Singham (2011). . Proceedings of the 2011 Winter Simulation Conference, 540-548.
Vieira Jr., H., S.M. Sanchez, K.H. Kienitz, and M.C.N. Belderrain (2011). . Proceedings of the 2011 Winter Simulation Conference, 3605-3616.
Vieira Jr., H., S. Sanchez, K.H. Kienitz, and M.C.N. Belderrain. 2011. Generating and improving orthogonal designs by using mixed integer programming. European Journal of Operational Research 215: 629638.
Shen, H., H. Wan, and S.M. Sanchez. 2010. A hybrid method for simulation factor screening. Naval Research Logistics 57(1): 4557.
Oh, R.P.T., S.M. Sanchez, H. Wan, M.E. Nissen, and T.W. Lucas, 2009, "Efficient experimental design tools for exploring large simulation models," Computational & Mathematical Organizational Theory 15(3): 237257.
Sanchez, S.M., H. Wan, and T.W. Lucas. 2009. Two-phase screening procedure for simulation experiments. ACM Transactions on Modeling and Computer Simulation 19(2): 124. DOI= http://doi.acm.org/10.1145/1502787.1502790
Cioppa, T.M., and T.W. Lucas. 2007. Efficient nearly orthogonal and space-filling Latin hypercubes. Technometrics 49(1): 4555.
Sanchez, S. M., F. Moeeni, and P. J. Sanchez. 2006. So many factors, so little time...simulation experiments in the frequency domain. International Journal of Production Economics, 103(1): 149165.
Sanchez, S. M. and P. J. Sanchez. 2005. Very large fractional factorials and central composite designs. ACM Transactions on Modeling and Computer Simulation 15(4): 362377.
Vinyard, W. and T. W. Lucas. 2002. Exploring combat models for non-monotonicities and remedies. PHALANX 35(1): 19, 3638.
General Simulation Analysis Topics
Felice, N., S. Shashaani, D. J. Eckman, and S. M. Sanchez (2024). .'' Proceedings of the 2024 Winter Simulation Conference, 3470-3481
Shashaani, S., D. J. Eckman, and S. M. Sanchez (2024). , ACM Transactions on Modeling and Computer Simulation, 34(4), Article 24, 24:1-24:29.
Taylor, S. J. E., C. Macal, A. Matta, M. Rabe, S. M. Sanchez, and G. Shao (2023). . Proceedings of the 2023 Winter Simulation Conference, 3296-3310.
Szabo, C., R. Castro, J. Denil, S. M. Sanchez, and P. Sloot (2023). . Proceedings of the 2023 Winter Simulation Conference, 660-670.
Pasupathy, R., D. I. Singham, and Y. Yeh. (2022). . Proceedings of the 2022 Winter Simulation Conference, 25-36.
Parmar, D., L. E. Morgan, A. C. Titman, R. A. Williams, and S. M. Sanchez (2022). . Proceedings of the 2022 Winter Simulation Conference, 97-108.
Taylor, S. J. E., Y.-J. Sun, J. Branke, O. Rose, and S. M. Sanchez (2021). ? Proceedings of the 2021 Winter Simulation Conference, 13 pages.
Parmar, D., L. E. Morgan, A. C. Titman, R. A. Williams, and S. M. Sanchez (2021). . Proceedings of the Operational Research Society Simulation Workshop 21 (SW21), 127-136.
Sanchez, S. M. and P. J. Sanchez (2020). , Proceedings of the 2020 Winter Simulation Conference, 60-74.
Lucas, T.W., W.D. Kelton, P.J. Sanchez, S.M. Sanchez, and B.L. Anderson. 2015. Changing the Paradigm: Simulation, Now a Method of First Resort. Naval Research Logistics 62: 293305.
Elmegreen, B. E., S. M. Sanchez, and A. Szalay. (2014). . Proceedings of the 2014 Winter Simulation Conference, 943-949.
Powers, M. J., S. M. Sanchez, and T. W. Lucas (2012). . Proceedings of the 2012 Winter Simulation Conference, 19291943.
Singham, D. and L.W. Schruben. 2012. Finite-sample performance of absolute precision stopping rules. INFORMS Journal on Computing, 24(4): 624-635.
Sanchez, P.J. and K.P. White. 2011.. Proceedings of the 2011 Winter Simulation Conference, 488-494.
More on Modeling Methodology
Sanchez, P. J. and S. M. Sanchez (2015). . Proceedings of the 2015 Winter Simulation Conference, 151-158.
Singham, D. and M. Thompson. 2011. . Proceedings of the 2011 Winter Simulation Conference, 2443-2449.
Buss, A.H. and A. Al Rowaei. 2010. A comparison of the accuracy of discrete event and discrete time. Proceedings of the 2010 Winter Simulation Conference, 14681477.
Sanchez, P. J. 2006. "As simple as possible, but no simpler: A gentle introduction to simulation modeling," Proceedings of the 2006 Winter Simulation Conference, 110.
Buss, A. H. and P. J. Sanchez. 2005. "Simple movement and detection in discrete event simulation," Proceedings of the 2005 Winter Simulation Conference, 9921000.
More on Applications
Singham, D. I. (2023). . Proceedings of the 2023 Winter Simulation Conference, 1054-1064.
Aros, S. K. and M. L. McDonald (2023). . Proceedings of the 2023 Winter Simulation Conference, 106-117.
Reich, D. and S. M. Sanchez (2023). . Naval Research Logistics, 70(7), 753-769.
Lovejoy, C. V., M. L. McDonald, T. W. Lucas and S. M. Sanchez (2021). "," 2021 Winter Simulation Conference (WSC), Phoenix, AZ, USA, 2021, pp. 1-12.
Parmar, D., L. E. Morgan, A. C. Titman, R. A. Williams, E. D. Regnier, and S. M. Sanchez (2021). . Proceedings of the 2021 Winter Simulation Conference, 12 pages.
Marlow, D., Sanchez, S. M., & Sanchez, P. J. (2019). Testing policies and key influences on long-term aircraft fleet management using designed simulation experiments. Military Operations Research, Vol 24 (3), 5 -25.
Thompson, C., T.W. Lucas, and M.L. McDonald (2020). Estimating the Effects of Combat Load Weight on Mission Outcomes, PHALANX, 53(1), 52-57. Winner of the Military Operations Research Societys 2021 John K. Walker Jr. Award for the best technical article published in PHALANX the previous year
Kesler, G., T. W. Lucas and P. J. Sanchez (2019). "," 2019 Winter Simulation Conference (WSC), National Harbor, MD, USA, 2019, pp. 2443-2454.
Langreck, J, H. Wong, A. Hernandez, M. L. McDonald, S. C. Upton, A. Pollman, and W. Hatch. (2019). Modeling and Simulation of Future Capabilities with an Automated Computer-Aided Wargame. Journal of Defense Modeling. Published online September 3, 2019.
Morgan, B. L., H. C. Schramm, J. R. Smith, T. W. Lucas, M. L. McDonald, P. J. Sanchez, S. M. Sanchez, and S. C. Upton (2017). Improving U.S. Navy campaign analysis using big data. Special Issue on Applications of Analytics and Operations Research in Big Data Analysis, Interfaces. Available online at
Upton, S. C., McDonald, M. L., Sanchez, S. M., Zabinski, H. M. (2017). . Proceedings of the 2017 Winter Simulation Conference. Piscataway, NJ: Institute of Electrical and Electronic Engineers, Inc.
MacCalman, A. D., Sanchez, S. M., M. L. McDonald, S. R. Goerger, and A. T. Karl (2016). . Proceedings of the 2016 Winter Simulation Conference, 3063-3074.
Marlow, D. O., S. M. Sanchez, and P. J. Sanchez (2015). Testing aircraft fleet management policies using designed simulation experiments. Proceedings of the 21st International Congress on Modelling and Simulation, 917--923. Available online at . Winner, 2015 Gus Schaefer Award, for the Best Paper at the Defence Operations Research Symposium (DORS), held concurrently with MODSIM2015, Gold Coast, Australia.
Hernandez, A., M. L. McDonald and J.Ouellet. (2015). Post Wargame Experimentation and Analysis: Re-examining Executed Computer Assisted Wargames for New Insights. Military Operations Research Journal, Volume 20, Number 4, 19-29.
McDonald, M. L., S. C. Upton, C. N. Seymour, T. W. Lucas, S. M. Sanchez, P. J. Sanchez, H. C. Schramm, J. R. Smith (2014). . Proceedings of the 2014 Winter Simulation Conference, 4136-4137.
Xing, D., H. Wan, M.Y. Zhu, S. M. Sanchez, and T. Kaymal (2013). "Simulation screening experiments using Lasso-optimal supersaturated design and analysis: A maritime operations application." Proceedings of the 2013 Winter Simulation Conference, Piscataway, New Jersey: Institute of Electrical and Electronic Engineers, Inc., 497508.
Kang, K., and M. L. McDonald. 2010. Impact of logistics on readiness and life cycle cost. Proceedings of the 2010 Winter Simulation Conference, 13361346.
Escher, L., S. Hall, E. Regnier, P. J. Sanchez, and J. Hansen. 2010. Simulating pirate behavior to exploit environmental information. Proceedings of the 2010 Winter Simulation Conference, 13301335.
Horne, G. E. and T. Meyer, 2010, Data farming and defense applications. MODSIM World Conference and Expo, "21st Century Decision-Making, The Art of Modeling & Simulation", Hampton Roads Convention Center, Hampton, VA, USA 1315 October 2010
Ahner, D. K., A. H. Buss, and J. Ruck. 2006. "Assignment scheduling capability for unmanned aerial vehicles - a discrete event simulation with optimization in the loop approach to solving a scheduling problem," Proceedings of the 2006 Winter Simulation Conference, 13491356.
Buss, A. H. and D. K. Ahner. 2006. "Dynamic allocation of fires and sensors (DAFS): A low-resolution simulation for rapid modeling," Proceedings of the 2006 Winter Simulation Conference, 13571364.
Kang, K., K. H. Doerr, and S. M. Sanchez. 2006. "A design of experiments approach to readiness risk analysis," Proceedings of the 2006 Winter Simulation Conference, 13321339.
Sanchez, S. M. 2006. "Work smarter, not harder: Guidelines for designing simulation experiments," Proceedings of the 2006 Winter Simulation Conference, 4757.
Lucas, T. W. and S. M. Sanchez. 2006. "The brave new world of designing simulation experiments for defense and homeland security applications," 2006 JSM Proceedings.
Forsyth, A. J., S. M. Sanchez, H. Wan, K. M. Chang, and P. J. Sanchez. 2006. "Exploring edge organizations for network centric operations," Proceedings of the 2006 CCRTS Conference.
Sanchez, S. M., H. Wan, and T. W. Lucas. 2006. "A two-phase screening procedure for simulation experiments" Proceedings of the 2005 Winter Simulation Conference, 223230.
Sanchez, S. M. 2005. "Work smarter, not harder: Guidelines for designing simulation experiments," Proceedings of the 2005 Winter Simulation Conference, 6982.
Cioppa, T. M., T. W. Lucas, and S. M. Sanchez. 2004. "Military applications of agent-based simulations," Proceedings of the 2004 Winter Simulation Conference, 171180.
Allen, T. E., A. H. Buss and S. M. Sanchez. 2004. "Assessing obstacle location accuracy in the REMUS unmanned underwater vehicle," Proceedings of the 2004 Winter Simulation Conference, 940948.
Sanchez, S. M. and H.-F. Wu. 2003. "Frequency-based designs for terminating simulation experiments: A peace-enforcement example," Proceedings of the 2003 Winter Simulation Conference, 952959.
Lucas, T. W., S. M. Sanchez, T. M. Cioppa, and A. I. Ipekci. 2003. "Generating hypotheses on fighting the global war on terrorism," Maneuver Warfare Science 2003, Marine Corps Combat Development Command, 2003, 117137.
Sanchez, S. M. and T. W. Lucas. 2002. "Agent-based simulations: Simple models, complex analyses," Proceedings of the 2002 Winter Simulation Conference, 116126.
Lucas, T. W., S. M. Sanchez, L. P. Brown, and W. Vinyard. 2002. "Better designs for high-dimensional explorations of distillations," Maneuver Warfare Science 2002, Marine Corps Combat Development Command, 1746.
Contact Information
SEED Center for Data Farming
261 Glasgow Hall
窪蹋勛圖厙
Monterey, CA 93943
Email: Contact SEED Center