Prevent duplicates when importing RSS feed to Core Data

Trying to import a RSS feed into Core Data. Once they are imported, when trying to update the feed again afterwards, how do I most efficiently prevent duplicates. Right now it checks every item against the datastore during the parsing, which is not very efficient.

I looked into the Top Songs sample from Apple. It uses a least recently used cache for categories. But when every item is different the cache doesn't help at all.

EDIT: To clarify, I can already identify each item uniquely in the feed with guid. The issue is the performance of comparing hundreds of items against the database every time, when most of them are duplicates.


When you are importing a new row you can run a query against the existing rows to see if it is already in place. To do this you create a NSFetchRequest against your entity, set the predicate to look for the guid property and set the max rows returned to 1.

I would recommend keeping this NSFetchRequest around during your import so that you can reuse it while going through the import. If the NSFetchRequest returns a row you can update that row. If it does not return a row then you can insert a new row.

When done correctly you will find the performance more than acceptable.

Can you modify your core data model ?

If you can I would add a "Hash" property to each feed entry to uniquely identify it. Then you could efficiently detect wether a specific entry is already in your database or not.

Category:iphone Time:2010-07-06 Views:0

Related post

  • Import RSS feed into Access 2013-03-22

    I would like to import or link to an RSS file, but I can't seem to do that. I thought that maybe I should try to create an Excel Object, but I am hitting a snag with that too

  • How do I import rss feeds from another computer? 2014-06-15

    I have some rss feed files from my outlook file on my old computer but the feeds did not come through when I opened the file. how do I get them back short of re entering the site addresses? --------------Solutions------------- After sending the quest

  • Import RSS feed into a Typo3 template 2010-06-09

    I'm a total beginner with Typo3 and would like to show a RSS feed in a Typo3 template using typoscript. And I have no idea how to do this ! Is there any way to do this quite easily ? Calling an external PHP script maybe ? Thx ! --------------Solution

  • Import rss feeds and usability 2010-08-12

    I run a small blog network and on this I have a page where I show the latest blog posts from different blogs on my server. I would like to extend this page, to also include new posts from external blogs using rss feeds. Currently it’s easy to get the

  • Duplicate posts from RSS feeds in Outlook 2012-04-04

    This issue was raised in 2010 and in 2012, but nothing appears to have been done about it. I can't find any solution posted in these forums. I'm still experiencing it today. And I see questions here from people who are using Outlook 2013 and experien

  • Importing RSS feeds 2013-01-22

    Hi, I am at a lost for importing RSS. Now when I do this using Access import xml I get two tables, but the problem is that there is no relation between these two tables and I am thinking that it would be easier to have the two tables merged into one.

  • Prevent Duplicates When Importing iCalendar files. 2014-01-08

    Is it possible to prevent Outlook 2007 from duplicating events that are imported in the iCalendar (.ics) format? I've seen reference to an option to prevent duplicates when using the Import/Export Wizard but I've been told that doesn't work on .ics o

  • Import RSS feed from LinkedIn? 2011-12-16

    Is there any way of getting the RSS feed of a LinkedIn group (like this: to show on a webpage? Would like to display the most recent discussion on a page. Thanks, --------------Solutions--------

  • Reusing a Django RSS Feed for different Date Ranges 2010-08-12

    What would be a way to have date range based rss feeds in Django. For instance if I had the following type of django rss feed model. from django.contrib.syndication.feeds import Feed from myapp.models import * class PopularFeed(Feed): title = '%s : L

  • Importing Complex XML into Core Data via NSXMLParser? 2010-06-30

    I've been working on importing XML into an iPad Core Data application. I have a working NSXMLParser implementation for my files, and have been able to import the simpler (ie attribute-only) elements into Core Data. Some of the XML dated has nested el

  • parsing get RSS feed with geotag data 2011-04-19

    I am still learning how to parse different XML feeds, so I was wondering how will one go about parsing an XML feed like the one below. is it possible to parse this without knowing the different XMl tags? Any steps to this, links or tutorial will be h

  • Checking for duplicate in sqlite before inserting them (Core data) 2011-08-04

    i'm inserting new objects into the database by core data. Is there any way to check if there is any duplicate in the database before i insert the values in? for (int i =0;i<[categoryArray count];i++) { Category * cat = [categoryArray objectAtIndex

  • Calendar RSS Feed - Sort by Date 2013-04-10

    I am wanting to use the RSS feed feature to display my calendar, but the dates are displaying in no chonological order when I select "Date" in the Sort by field. Has anyone found a way to get this to work? --------------Solutions------------- In orde

  • How should I import object instances into Core data? 2012-02-05

    I created a custom LocationGenerator class that uses CoreLocation and Reverse Geocoding, and generates (when asked) a custom Location object. My custom Location object has two instance variables - Address and GPS...both point to instance of two custo

  • Prevent duplicates when importing from Excel 2014-06-27

    I have a process where the Access admin periodically* runs two different reports (RANT and NRA) from a larger database and exports them into two Excel files (RANT.xls and NRA.xls). They then remove a blank header row (weird report formatting) from ea

  • Alternative to preventing duplicates in importing CSV to CouchDB 2011-05-08

    I have 2 big CSV file with millions of rows. Because those 2 CSVs are from MySQL, I want to merge those 2 tables into one Document in couch DB. What is the most efficient way to do this? My current method is: import 1st CSV import 2nd CSV To prevent

  • Importing RSS feeds into Excel 2013 - XML Mapping problem 2014-08-20 This article seemed pretty good until I downloaded the file and tried it out for myself. The problem relates to t

  • Average Size of an RSS/Feed file, for Data Storage and Bandwidth Calculation 2009-10-25

    Doing a back of the envelope calculation to determine network bandwidth and data storage needed to monitor approx 10,00,000 feeds every 20 minutes. Any idea what could be the average size of an rss file ? I remember reading somewhere the guys from te

  • Django1.2 rss feed descriptions and dates - Google Reader 2011-06-20

    The docs state all you all to do is define the description and date. This does not work. the descriptions and and dates are not output in the feed. Only the title. Does anyone know how to output the description and date? all help is greatly appreciat

Copyright (C), All Rights Reserved.

processed in 0.706 (s). 13 q(s)