Talk:SemEval

{hang on}

Now the SemEval wiki page has sufficient content to identify with the subject. The sudden deletion of content was an attempt to wikify the page but i've undone the deletion

" 00:45, 9 November 2010 Alvations (talk | contribs) (19,568 bytes) (Please advise if this deletion is necessary, I've created and ported the details of individual workshops to their own wiki page) (undo)"

Please check and verify the current page before considering a page deletion. —Preceding unsigned comment added by Alvations (talk • contribs) 17:00, 8 November 2010 (UTC)

harmonizing tense

I think you need to do some work on harmonizing tense in the descriptions of the various semevals --- you are using present where I think past is appropriate. Francis Bond (talk) 07:41, 7 November 2010 (UTC)

why wikify

Responding to a comment User_talk:Jodi.a.schneider#Semeval here:

Mainly I'm trying to draw attention of some more experienced wiki-folk. There were no simple, obvious changes I could think of. I thought about both wikify or "reads like an essay", even though neither of those is spot-on.

As far as the improvements I suggest:

• Simplify the language in the introduction. Give more context and start with the most important things first: it's a series of workshops aimed at a particular NLP problem. Then explain the problem.
• Consider who the audience is.
• Make WP:Notablity obvious. Are there reviews or writeups about the workshop? What makes this article of general interest, rather than appropriate for an NLP or CL encyclopedia?
• Reduce the use of tables, if it's reasonable to do so. They're a pain to look at on my laptop; it would be even worse on a mobile device.
• Consider where the appropriate places are for historical information (e.g. Senseval, DARPA's former name, etymology of the workshop name).
• Consider using citation templates like Template:Cite journal; they make it easier for people to extract references into bibliographic management software.

I hope this helps, and if you want more feedback on a later version, or more details about what I mean, feel free to ping me. Jodi.a.schneider (talk) 15:16, 8 November 2010 (UTC)

In addition: There are way too many in-line external links - this is frowned upon by the relevant guideline. There are also Wikilinks in section headings - also a no-no per the manual of style. Finally (for now at least) the content of some of the sections is duplicated in separate articles such as Senseval-3. These separate articles have all been nominated for deletion, see the discussion at Wikipedia:Articles for deletion/SemEval-1.  – ukexpat (talk) 18:15, 8 November 2010 (UTC)

Stuff I removed

Trying to do a little bit on page improvement, so I removed this section. Jodi.a.schneider (talk) 00:51, 9 November 2010 (UTC)

==="-Eval" Etymology===

"-Eval" is a fairly recent morpheme for conferences, workshops and algorithms related to computational evaluations. The "-Eval" innovation originate from the evaluation metric for computational grammar systems. Grammar Evaluation Interest Group (GEIG) evaluation metric, also termed as the Parseval metric ,, a blend of grammatical "pars"ing and system "eval"uation. Progessively, a series of well intended puns motivates the popular use of the "-eval" morpheme:

• Parseval's (commonly spelled as Percival), one of King Arthur's legendary Knights of the Round Table, involvement in the quest for the holy grail symbolizes computational linguists' ultimate quest for computer to understand natural language.

• Parseval coincides with the Parseval theorem (a fourier series related theorem that most computer scientists are familiar with).

And these sentences Jodi.a.schneider (talk) 00:57, 9 November 2010 (UTC)

Computational semantic analysis is a composite of the "Semantic Analysis" and the "Computational" components. "Semantic Analysis" refers to a formal analysis of meaning, and "computational" refer to approaches th in principle support effective implementation..

I put this info into the lede, without the link.Jodi.a.schneider (talk) 01:11, 9 November 2010 (UTC)

ACL-SIGLEX (Special Interest Group on the LEXicon of the Association for Computational Linguistics)is the umbrella organization for SemEval semantic evaluations and the SENSEVAL word-sense evaluation exercises.

Further pared down, removing this Jodi.a.schneider (talk) 01:14, 9 November 2010 (UTC)

The first three evaluation workshops, Senseval-1, Senseval-2 and Senseval-3, were focused on Word Sense Disambiguation Systems (WSD). More recently, Senseval had become SemEval, a series of evaluation exercises for semantic annotation involving a much larger and more diverse set of tasks . Beginning with the 4th workshop, SemEval-1, the nature of the tasks evolved to include semantic analysis tasks outside of word sense disambiguation.

The framework of the SemEval/Senseval evaluation workshops emulates Message Understanding Conferences (MUCs) and other evaluation workshops ran by ARPA (Advanced Research Projects Agency, renamed the Defense Advanced Research Projects Agency (DARPA)).

SemEval Framework, adapted from MUC introduction

Stages of SemEval/Senseval evaluation workshops

1. Firstly, all likely participants were invited to express their interest and participate in the exercise design.
2. A timetable towards a final workshop was worked out.
3. A plan for selecting evaluation materials was agreed.
4. 'Gold standards' for the individual tasks were acquired, often human annotators were considered as a gold standard to measure precision and recall scores of computer systems. These 'gold standards' are what the computational systems strive towards. (In WSD tasks, human annotators were set on the task of generating a set of correct WSD answers(i.e. the correct sense for a given word in a given context)
5. The gold standard materials, without answers, were released to participants, who then had a short time to run their programs over them and return their sets of answers to the organizers.
6. The organizers then scored the answers and the scores were announced and discussed at a workshop

References

Differentiating from WSD page

Hi Jodi, thanks for cleaning up the page, but i would like to raise the point about differentiating SemEval page from the WSD page. If you left out explanation of the how and what the workshop is about, then the information will be the same as the ones on the WSD page which will go get into the problem with WP:notability. —Preceding unsigned comment added by Alvations (talk • contribs) 04:06, 9 November 2010 (UTC)

This page already has a problem with WP:notability: on Wikipedia it's necessary to assert why a topic belongs in this general-purpose encyclopedia. As someone at the AfD has suggested, to show notability of the workshops general coverage of them "in non-scholarly sources such as newspapers or semi-scholarly work such as Science Daily or Chronicle of Higher Education" is necessary. Does that make sense? Jodi.a.schneider (talk) 10:24, 9 November 2010 (UTC)

Detailed tasks

I think the detailed tasks are probably too much for a Wikipedia article; it's even worse to have them in separate pages. The main purpose of a page about the workshop should be to explain it to laymen. Perhaps there could be references to published summaries (i.e. conference webpage or formal publication) of the task descriptions instead? Jodi.a.schneider (talk) 10:27, 9 November 2010 (UTC)

Sadly the conference webpage links to detailed papers, no one published summaries of the workshop, they have summaries of how they fare in the task. If i were to create a summary it might be "original work". The ccnference webpage looks like the wiki sections, i.e. a list of the task and how they are carried out. Alvations (talk) 23:21, 9 November 2010 (UTC)

lede

This change reintroduced several errors. Please spell-check the lede. I'm leaving it alone for awhile, especially since more editors are likely to see the rescue notice. I see that you're worried about duplicating Word Sense Disambiguation; I'd worry less about that than about utterly confusing readers in the first paragraph. Jodi.a.schneider (talk) 16:42, 9 November 2010 (UTC)

thank you jodi

Just saw the changes you made for the page, thanks! now it's kind of more wikified than it started off. —Preceding unsigned comment added by Alvations (talk • contribs) 18:17, 11 November 2010 (UTC)

Trimmed page -- proposed

I've put up a draft of a trimmed article: Talk:SemEval/Proposed_Revision. What do you think? It's an attempt to make this article more similar to Message Understanding Conference and Text Retrieval Conference. Jodi.a.schneider (talk) 19:34, 14 November 2010 (UTC)

Hi Jodi, the version that you've proposed is not the best template to follow. It would have demoted the page to a stub, which isn't a solution to rescue the current page.Alvations (talk) 05:50, 16 November 2010 (UTC)

Deletion - no consensus, so small suggestion

This page seems to hold for WP:notability, and now the page seems more wikified to the general wiki readership. i am very tempted to put the Senseval-1 to SemEval-2 sections back on this page from my user-subpage. please discuss/comment about the suggestion, otherwise i will get into the whirlpool of "adf" and "rescue" and "wikifiable" and other recondite rules of wiki.

I think it's too much detail. Jodi.a.schneider (talk) 15:30, 17 November 2010 (UTC)

Thanks Jodi, hopefully the people who's looking for information will source their way out of to the external links.Alvations (talk) 15:41, 17 November 2010 (UTC)

There should definitely not be links to the individual events inside the article body. Jodi.a.schneider (talk) 22:05, 17 November 2010 (UTC)

I disagree. WP:EL specifically allows for Sites that contain neutral and accurate material that is relevant to an encyclopaedic understanding of the subject and cannot be integrated into the Wikipedia article due to copyright issues, amount of detail (such as professional athlete statistics, movie or television credits, interview transcripts, or online textbooks), or other reasons (WP:ELYES). The links to the individual event pages are definitely neutral (as far as I know there is no controversy associated with SemEval) and accurate and have too much detail for this page. Having them linked in the body makes it easy for readers to get more information if they want it, which is one of the great advantages on of an online encyclopaedia. AT the least, I think they should be in the External Links section (I know they are in the template, but wiki novices may not find them there). Francis Bond (talk) 23:50, 17 November 2010 (UTC)

Notability challenge again

I question the need to re-challenge the notability of the SemEval, given that it has gained much attention in the bid for deletion previously http://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/SemEval. This wikipedia article had also been referred to as the encylopedic source for the SemEval conference from the recent conference page. http://www.cs.york.ac.uk/semeval-2012/ Alvations (talk) —Preceding undated comment added 10:31, 25 October 2012 (UTC)

Wikipedia's policy on notability requires "significant coverage in reliable sources that are independent of the subject". Currently this page cites scientific papers that came out of SemEval or Senseval tasks, work groups, or conferences. Those sources are not independent of SemEval, and so they do not establish notability per Wikipedia's standard. Mentions of the project as a project in sources excluding the work of project participants need to be found and cited. I would recommend looking at something like university newsletters or local newspapers from host cities rather than scholarly work. Cnilep (talk) 04:30, 26 October 2012 (UTC)

Ongoing conferences

See https://semeval.github.io/ - may have useful information for updating article. --SarekOfVulcan (talk) 19:45, 1 November 2022 (UTC)