The Core Data sqlite-wal file gets MASSIVE (> 7GB) when inserting ~ 5000 rows

存储架构 2017-11-21 阅读原文

I'm importing data into Core Data and find that the save operation is slow. Using the iOS simulator, I watch the sqlite-wal file grow and grow until its over 7GB in size.

I'm importing approx 5000 records with about 10 fields. This isn't a lot of data.

Each object I'm inserting has a to-one relation to various other objects (6 relations total). All of those records combined equal less than 20 fields. There are no images or any binary data or anything that I can see that would justify why the resulting size of the WAL file is so huge.

I read the sqlite docs describing the wal file and I don't see how this can happen. The source data isn't more than 50 MB.

My app is multi-threaded. I create a managed object context in the background thread that performs the import (creates and saves the core data objects).

Without writing the code out here, has anyone encountered this? Anyone have a thought on what I should be checking. The code isn't super simple and all the parts would take time to input here so lets start with general ideas.

I'll credit anyone who gets me going in the right direction.

Extra info:

  • I've disabled the undo manager for the context as I don't need that (I think it's nil by default on iOS but I explicitly set it to nil).
  • I only call save after the entire loop is complete and all managed objects are in ram (ram goes up to 100 MB btw).
  • The loop and creation of the core data objects takes only 5 seconds or so. The save takes almost 3 minutes as it writes the the awl file.

It seems my comment to try using the old rollback(DELETE) journal mode rather than WAL journal mode fixed the problem. NOTE that there seem to be a range of problems when using WAL journal mode including the following:

  • this problem
  • problems with database migrations when using the migratePersistentStore API
  • problems with lightweight migrations

Perhaps we should start a Core Data WAL problems page and get a comprehensive list and ask Apple to fix the bugs.

Note that the default mode under OS X 10.9 and iOS 7 now uses WAL mode. To change this back add the following option

@{ NSSQLitePragmaOptions : @{ @"journal_mode" : @"DELETE" } }
Hello, buddy!

责编内容by:Hello, buddy!阅读原文】。感谢您的支持!


Metal入门教程(一)图片绘制 前言 这里是一篇Metal新手教程,先定个小目标:把绘制一张图片到屏幕上。 Metal系列教程的 代码地址 ; OpenGL ES系列教程在这里; 你的star和fork是我的源动力,你的意见能让我走得更远。 核心思路 通过MetalKit, 尽量简单地 实现...
iOS多线程:『GCD』详尽总结(二):GCD 的基本使用... 同步执行 + 并发队列 在当前线程中执行任务,不会开启新线程,执行完一个任务,再执行下一个任务。 /** * 同步执行 + 并发队列 * 特点:在当前线程中执行任务,不会开启新线程,执行完一个任务,再执行下一个任务。 */ - (void)syncConcurrent...
iOS 代码的 taste(品味) 最近看了不少代码,想起写代码有意思的地方之一在于,实现同一个 feature,修复同一个 bug,不同程序员可以写出风格迥异的代码,甚至流程也不同,虽然最后都可行,从结果论的角度对用户来说是一致的。我们可以称这种差异为个人 taste,taste 有好坏高低之分,但有时候如何评定却很难有一个清晰准确...
Questions about buttons in MonoTouch I'm working with MonoTouch and I have come up with three questions around buttons that I'm hoping someone can help me with. I'm trying t...
iOS开发细节 | 通知怎么写? 前言 工作的这一年多时间里我见识了各种通知的写法,今天总结一下。 第一种: 最纯粹的,如: postNotificationName:@"aNotification" object:nil]; 第二种: 在PCH文件中写宏,如: #define aNo...