Discussions on Loading Data and Contributing Large Datasets

No, that's not quite true.  We'll respond to requests for quota increases whatever forum you ask for them in.  You just have to ask clearly and we'll see it, whether it's here in the discussions, on the dev list, or on Jira.

Note that it may take us a few days to get to it, and that we will want to review your work before increasing your quota.  

I asked this question on the developer's email list and was told that set of queries below would return the information needed (replace 'tfmorris' with your username and adjust the dates appropriately), although I found the results to be off by >25% and I'm not sure why.

I was also told:

- it's a rolling 24-hour clock, so no fixed reset time

- the sandbox isn't currently write throttled  

tfmorris","return":"count"},"as_of_time":"2009-02-01"}}' 

The data looks good!

However, I lied when I said earlier, on the mailing list, that the IMDB Profile Page model was the way to go. We now have the ability to use keys into an external database as a way to generate URIs, which further provides uniqueness checking. I am working on converting our IMDb references into this form. It would be great if you could wait on the final NNDB load and model it that way; I will be happy to show you how once I figure it out myself. (-:

Oh, yeah! It’s totally done and I forgot to come back here.

First, I made namespaces (/authority/imdb/film, name, character). Then I made properties that enumerate those namespaces and attached URI templates to them.

Check out the IMDb profile property on film. It expects Enumeration as its type. Then you have to get a little fancy and switch to the admin view. Set the Enumeration property of the property to the expected namespace. Then co-type the property as a Foreign key property and set a URI template. Set its type to URI Template and fill in both the canonical template (used to generate and recognize URIs) and the other templates (used only for recognition).

The schema UI will support this at some point… just not yet, as it’s kind of a power-user thing.

Ok, I've created the namespace, I've set up the enumeration property on the NNDB Person type and I've attached a URI template to that property.

Then I added a sample key to the Paul Newman topic and the NNDB link shows up as expected. Unfortunately, the ID has a forward slash in it which gets escaped and breaks the link. I went back to the Foreign Key Property and explicitly disabled URI encoding but that hasn't fixed it. Any ideas on how to handle this?

Coincidentally, tsegaran added a foreign key to a NYT page on the same Paul Newman topic and his key also contains forward slashes but he seems to have entered the weblink seperately without using a URI template.

Ah, yes, the char escaping with keys and URI templates... I am in the process of converting the NYT keys to use URI templating, and I also ran into that problem.  There's a bug filed to have the UI behave properly when it encounters escaped URLs - I'll post back when there's a status update to this.

Toby (tsegaran) actually added a key, and created a discrete weblink (it's not using URI templating).  When I implement the URI templating, I'll be removing the superfluous weblink.

BTW, good work!

I've uploaded a new version of the data to the sandbox. If no one has any objections, I'll add it to the main site.

Is there a limit on how many writes I can do on the main site? Will I be able to make 19,000 writes in a day?

Hi Ben,

I've passed your request on to the data team - someone should be replying to you shortly.

Hey Ben -

Are working with Benjamin Good on this?  I thought his original idea for using Freebase to annotate Connotea was great. I am very supportive of using Freebase IDs as content tags.

Could you send me a note about the nature of the candidate tags.  Are they tied to other source vocabularies we could reference?  I'd also be curious if there are other semantic relationships we could draw out between the tags (or other Topic vocabularies) to make navigating the annotated relationships in Connotea more powerful.

Jamie [at] metaweb

Hi Jamie, 

Wow, you know about ED already.  Cool! 

 Yes, I am working with Ben Good on this experiment.   Basically, what he is doing is using ED to annotate BioMOBY web services with Freebase Topics (each BioMOBY web service has a unique URI).  In addition to general tags to describe a service, he is also collecting a special set of tags which describe the relationship between the input and output data: i.e.  "Each output is a about/of the input(s)", e.g. "Each output is a _Homolog_ about/of the input(s)"

Since the services are all bioinformatics related, my plan was to load terms from a biology/bioinformatics ontology into Freebase.   My choice is the NCI Thesaurus (http://ncicb.nci.nih.gov/xml/owl/EVS/Thesaurus.owl) , which has about 80,000 classes representing all sorts of biological things.

The main reason we want to use Freebase for this is because of the great type-ahead feature.    I was planning to keep it simple and just load all the terms under one Type.  So to be honest, it's "throw away" data, that I would remove from Freebase after the experiment was done.

Thanks for your interest :-)

 - Ben Vandervalk

Oh also, you can browse the NCI Thesaurus at http://bioportal.bioontology.org/visualize/13578

 - Ben V.

Hi Ben,

I've been working on some ontology mapping types based off of some of Jamie's own Web Ontology types. If you import 80,000 topics into Freebase I would be really interested in seeing those Freebase topics mapped back to their original URIs in the NCI Thesaurus.

To give you some idea of how this would look, I have a basic Freebase to FOAF mapping here. 

Shawn 

Cool, nice work :-) That's the proper way to do it, I think.

My intention is to something quick and dirty though, just so I can use the Freebase type ahead feature in our annotation experiment.   Mapping all 80,000 classes and umpteen properties to existing Freebase entities would be a ton of work, and I just don't have time to do that!

- Ben V.

Hi Ben,

Have you been communicating with Jamie about this?  Can you send him an email regarding the nature of the candidate tags - his email is jamie [at] metaweb.  Thanks.

Sure, I'll copy my message above to jamie@metaweb.com.

-Ben V.

Hi cooksey87,

 I've passed this information on to our data team and someone should be getting back to you shortly!

Is Schiphol actually contained by Amsterdam?  I am not that familiar with Dutch location containment, but it seems like Schiphol is contained by the municipality of Haarlemmermeer and possibly contained by the city Schiphol-Rijk.

Are you just adding containment by country, or containment by administrative division also?  If it is also the latter, there could also be the situation where an airport may serve one location, but be contained by a different administrative division.  The example I'm thinking of is Newark Airport is contained by New Jersey, but serves New York City.

You have a point, I think there is no airport that is actually in the city they serve, it would be just outside the city. But nobody says thei're flying to schiphol rijk or whatever (unknown small) town that is closest to the airport or containing the airport.

I guess thats why serves is there. In case of Schiphol, the administrative division Amsterdam is responsible for Schiphol. 

I will only add the countries containing the airport.

The types you should be looking at are /education/institution and /education/university. Between them, they should have all the properties for post-secondary schools. (Educational Institution has properties that are common to schools of all levels.)

If there is data in your dataset that you want to load,and which doesn't match any existing properties, we can talk about adding new properties. The first place to start, though, would be to create a type in your private domain to hold those properties so you can test them out and so that people can review them before adding them to the types in the education domain.

This looks like a great dataset. Feel free to keep posting questions while you work on it.

Link is broken or leads to nothing?! is there any solution adding recipes in freebase?

sebastian 

Hi Sebastian,

User skud was looking for reviewers on her food domain.  I would think that would be a good start for then developing a type for recipes where it could use those types.  Perhaps you may want to collaborate with other users on creating a recipe type?  You could start a discussion on the data-modeling list (if you're not part of the list, you can join here).

Thanks, Jeroen. First, please be sure that the data can be contributed legally; we did not collect the artist similarity information from MusicBrainz, for instance, because it is licensed CreativeCommons-Non-Commercial, and as a commercial enterprise, we can’t legally use that data.

If you have permission to load that data, then this sounds like a great idea! You could also add the last.fm page as a Web link. For lookup, since last.fm and Freebase both use MusicBrainz keys, a key is probably not needed