JAIST Repository >
School of Transdisciplinary Science >
Articles >
Journal Articles >

Please use this identifier to cite or link to this item: http://hdl.handle.net/10119/18245

Title: Generation of Game Stages with Quality and Diversity by Reinforcement Learning in Turn-based RPG
Authors: Nam, SangGyu
Hsueh, Chu-Hsuan
Ikeda, Kokolo
Keywords: Reinforcement learning
procedural content generation
turn-based rpg
machine learning
Issue Date: 2022-09
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Magazine name: IEEE Transactions on Games (ToG)
Volume: 14
Number: 3
Start page: 488
End page: 501
DOI: 10.1109/TG.2021.3113313
Abstract: Many recent studies in procedural content generation (PCG) are based on machine learning. One of the promising approaches is generative models, which have shown impressive results in generating new pictures and videos from existing ones. However, it is usually costly to collect sufficient content for training on PCG. To address this issue, we consider reinforcement learning (RL), which does not need to collect training data in advance but learns from its interaction with an environment. In this work, RL agents are trained to generate stages, which we define as series of events in turn-based role playing games (RPG). It is a challenging task since several events in a stage are usually highly correlated to each other. We first formulate the stage generation problem into a Markov decision process. A handcrafted evaluation function, which simulates players’ enjoyment, is defined to evaluate generated stages. Two RL algorithms are selected in the experiments, which are deep Q-network (DQN) for discrete action space and deep deterministic policy gradient (DDPG) for continuous action space. The generated stages from both models receive evaluation values indicating good quality. To solve the delayed reward problem and further improve the quality of the stages, we employ virtual simulations to give rewards to intermediate actions and get stages with higher average scores. In addition, we introduce noise to avoid generating similar stages while trying to keep the quality as high as possible. The proposed methods succeed in generating good and diverse stages.
Rights: This is the author's version of the work. Copyright (C) 2022 IEEE. IEEE Transactions on Games, 14 (3), 2022, 488-501. DOI: 10.1109/TG.2021.3113313. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
URI: http://hdl.handle.net/10119/18245
Material Type: author
Appears in Collections:d10-1. 雑誌掲載論文 (Journal Articles)

Files in This Item:

File Description SizeFormat
I-IKEDA-K0405-22.pdf4989KbAdobe PDFView/Open

All items in DSpace are protected by copyright, with all rights reserved.


Contact : Library Information Section, Japan Advanced Institute of Science and Technology